-- Logs begin at Tue 2022-08-09 17:06:25 UTC, end at Tue 2022-08-09 18:18:37 UTC. -- Aug 09 17:42:06.112792 dv-cinder-836083-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 17:42:06.126392 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 17:42:06 2022] *** Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: nodename: dv-cinder-836083-11-iscsi Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: machine: x86_64 Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: clock source: unix Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: pcre jit disabled Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: detected number of CPU cores: 4 Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: current working directory: / Aug 09 17:42:06.128459 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: detected binary path: /usr/bin/uwsgi-core Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: your processes number limit is 47680 Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: your memory page size is 4096 bytes Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: detected max file descriptor number: 2048 Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: lock engine: pthread robust mutexes Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: thunder lock: enabled Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 17:42:06.128979 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 17:42:06.168336 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: Python main interpreter initialized at 0x55c1fe76e9c0 Aug 09 17:42:06.168336 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: python threads support enabled Aug 09 17:42:06.168336 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: your server socket listen backlog is limited to 100 connections Aug 09 17:42:06.168336 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: your mercy for graceful operations on workers is 90 seconds Aug 09 17:42:06.168738 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 17:42:06.168816 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: *** Operational MODE: preforking *** Aug 09 17:42:06.169033 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: *** uWSGI is running in multiple interpreter mode *** Aug 09 17:42:06.169033 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: spawned uWSGI master process (pid: 80201) Aug 09 17:42:06.169242 dv-cinder-836083-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 17:42:06.172533 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: spawned uWSGI worker 1 (pid: 80212, cores: 1) Aug 09 17:42:06.172533 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: spawned uWSGI worker 2 (pid: 80213, cores: 1) Aug 09 17:42:06.172533 dv-cinder-836083-11-iscsi devstack@keystone.service[80201]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 17:42:11.680555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:42:11.680555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: from cryptography.utils import int_from_bytes Aug 09 17:42:11.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:42:11.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: from cryptography.utils import int_from_bytes Aug 09 17:42:11.682241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:42:11.682241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Aug 09 17:42:11.682704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 17:42:11.682704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Aug 09 17:42:11.732311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 17:42:11.732532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 17:42:11.732727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] command line args: [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 17:42:11.732998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 17:42:11.733175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ================================================================================ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 17:42:11.733335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 17:42:11.733443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] admin_token = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.733530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 17:42:11.733688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_dir = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.733772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] command line args: [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 17:42:11.733877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] config_source = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 17:42:11.734224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] debug = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ================================================================================ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 17:42:11.734426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_dir = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.734964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_source = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] debug = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.735980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_config_append = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_dir = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.736957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_config_append = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_dir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.737947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.738676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.739981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.740906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] publish_errors = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.741923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] standard_threads = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.742966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_journal = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_json = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_stderr = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.743995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] use_syslog = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] publish_errors = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.744528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.744728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.744812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.745054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.745134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.745304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.745382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.745544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.745667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] standard_threads = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.745800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.745975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.746053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.746272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.746349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.746510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.746587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.746719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_journal = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.746798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.746926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_json = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.747036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.747157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_stderr = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.747237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.747355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_syslog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.747464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.747656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 17:42:11.747734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.747962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.748945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.749956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.750894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.751887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.752856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.753996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.754897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.755907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.756063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.756805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.757957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.758918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.759895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.760981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.761951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.762985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.763950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.764886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.765847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.766951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.767989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.768933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.769869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.external = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.770856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.password = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] auth.token = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.771971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.772970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.773923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.774081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.774168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.774442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.external = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.775924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.password = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.token = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.776983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.777935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.778897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.779878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.780999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.781970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.782986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.783971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.784954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.785890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.786829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.787965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.788980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.789936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.790858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.791847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.792975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.793994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.794918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.795896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.796989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.797943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.798961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.799914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.800870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.801032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.801111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.801774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.801880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.802939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.803940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.804880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.805912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.806945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.807945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.808899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.809948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.810940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.811919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.812996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.813977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.814992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.815976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.816824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.817546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.817637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.817783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.817863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.818874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.819877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.820988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.821235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.821486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.821721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.821965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.822262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.822524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.822771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.823048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.823289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.823538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.823776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.824015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.824251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.824548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.824872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.824963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.825992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.826082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.826340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.826428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.826590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.826753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.827118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.827224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.827389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.827507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.827595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80212) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 17:42:11.827872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.828274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 17:42:11.828481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 17:42:11.857993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.858228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.858460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.858604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.858839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.859100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.868423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.868858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.897229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.897439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.897534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.897674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.950072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:11.950239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.001396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.001631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.002026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.002265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.067941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.068318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.092973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.093645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.105227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.105662 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.118880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80212) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.119929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 17:42:12.136803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.137014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:42:12.137356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.137441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.137558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:42:12.137661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:42:12.137837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.137922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 17:42:12.138368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.138889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.139183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.150679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.150957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.151859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.158448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.158731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.161639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.161854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.161940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.162799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.163925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.164102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.164186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.164372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.164454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.164642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.187727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.187830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.189253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.189349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.189538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.189636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.194710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.194802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.195018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.195104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.195308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.195404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.201950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.202050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.202191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.202331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.206211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.206371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.206506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.206665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.206789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.206948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.207096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.207255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.207371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.208898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.224377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.224653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.224738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.224934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.228731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.228992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.232154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.232443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.232529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.232742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.236827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.237152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.240045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.240373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.243209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.243506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.243641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.243793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.243946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.244822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.245103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.261058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.261346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.261617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.261706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.261880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.261984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.262245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.262515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.268729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.269523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.270861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.271163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.271684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.271966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.273953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.274247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.274544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.274778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.274869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.275089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.275181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.275389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.275683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.275960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.286512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.286797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.287107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.287390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.287491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.287671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.287763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.287944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.288034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.288216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.288315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.288533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.288808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.289086 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.304826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.306009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.307808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.309012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.310726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.311999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.312339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.313576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.313819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.314103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.314396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.315104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.315393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.315678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.321393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.322745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.323098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.324461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.326029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.326321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.326603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.326887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.327438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.327735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.328023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.328302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.335586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.335890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.336248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.337283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.337509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.337792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.343135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.343419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.343701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.343978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.344251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.344531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.344699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80212) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 17:42:12.344829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.344987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.345966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.346842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.347000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.347233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.347507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.347781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.348080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.348347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80212) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 17:42:12.380828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c1fe76e9c0 pid: 80213 (default app) Aug 09 17:42:12.382331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c1fe76e9c0 pid: 80212 (default app) Aug 09 17:42:12.384127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307afd9-33e0-4730-838b-84276bb85544 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:12.384414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307afd9-33e0-4730-838b-84276bb85544 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:12.384688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307afd9-33e0-4730-838b-84276bb85544 None None] PATH_INFO: `/v3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:12.389038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Tue Aug 9 17:42:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:20.126788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971b1f-c6c8-429c-9205-734984580533 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:20.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971b1f-c6c8-429c-9205-734984580533 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:20.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971b1f-c6c8-429c-9205-734984580533 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:20.131838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:20.137097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:20.137528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:20.137645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:20.213516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: INFO dbcounter [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] Registered counter for database keystone Aug 09 17:42:20.233881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG oslo_db.sqlalchemy.engines [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80213) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 17:42:20.240319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writer thread running {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 17:42:20.419505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80213) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 17:42:20.419922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80213) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 17:42:20.474161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:20.475793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:20.609558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 17:42:20.754099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05971b1f-c6c8-429c-9205-734984580533 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:20.837138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: INFO dbcounter [None req-05971b1f-c6c8-429c-9205-734984580533 None None] Registered counter for database keystone Aug 09 17:42:20.865500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG oslo_db.sqlalchemy.engines [None req-05971b1f-c6c8-429c-9205-734984580533 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80212) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 17:42:20.871991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writer thread running {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 17:42:20.997202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05971b1f-c6c8-429c-9205-734984580533 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:21.001584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:21.002236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:21.002516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:21.002789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:21.012459 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:21.070482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING keystone.common.rbac_enforcer.enforcer [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 17:42:21.070614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c421cc-eb9d-4322-bff9-68557fe2fb4c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:21.080962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Aug 9 17:42:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 17:42:21.086816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:21.102215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dd531-b0ee-4882-a745-5bce1402f2ea None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:21.106824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:21.108010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:21.108287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:21.108558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] PATH_INFO: `/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:21.109988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] RBAC: Authorizing `identity:get_project(project_id=5aa4b373e99e4d74b2148fd834c1abbb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:21.161659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 17:42:21.162289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150482e-967f-424e-89e5-592f357bef08 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:21.166885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Tue Aug 9 17:42:21 2022] GET /identity/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:23.165044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952c3c-15ef-4c71-b55a-50059fcec3b7 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:23.165584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952c3c-15ef-4c71-b55a-50059fcec3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:23.165584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952c3c-15ef-4c71-b55a-50059fcec3b7 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:23.169708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:23.175289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:23.175763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:23.176190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:23.198852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:23.200450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:23.209290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 09 17:42:23.232535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952c3c-15ef-4c71-b55a-50059fcec3b7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:23.244598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952c3c-15ef-4c71-b55a-50059fcec3b7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:23.248717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:23.249313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:23.249591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:23.249863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:23.257714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:23.259602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d94c1-40dc-4d50-a06f-5b45e12f454d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:23.268586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Aug 9 17:42:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:42:23.275065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:23.284915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65ddf-875a-4b87-bf82-ee68e10b0fbf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:23.290311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:23.290951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:23.291305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:23.291560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:23.293037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:23.294874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cc844-c14c-427b-bae0-1412e49fa8d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:23.296480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Tue Aug 9 17:42:23 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:25.392198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537c70-ef71-40af-a1f4-ef8dda19b14c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.392748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537c70-ef71-40af-a1f4-ef8dda19b14c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.392838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537c70-ef71-40af-a1f4-ef8dda19b14c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.394096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:25.399669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.399669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.399669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.416614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6278e-a5d6-47d0-9c61-74aeb72d5f94 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.417097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6278e-a5d6-47d0-9c61-74aeb72d5f94 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.417491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6278e-a5d6-47d0-9c61-74aeb72d5f94 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.419816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:25.420099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:25.421451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:25.427259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.427259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.427259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.469995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG passlib.handlers.bcrypt [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80212) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 17:42:25.470427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG passlib.handlers.bcrypt [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80212) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 17:42:25.507395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:25.517187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:25.518783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:25.530189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:25.554397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:25.627390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:25.646325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c93fd-9e58-417b-8905-74086e5619ab None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:25.649089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:25.681576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:25.682209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.682489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.682767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.694046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:25.702763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:25.703938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:25.704815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:25.709646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:25.753112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822221-b85c-4f21-9a80-e0dee367be6f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:25.755423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:25.791506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:25.792371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.792655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.792930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.794025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:25.816162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8746-c7df-4021-a866-c58fca04968f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:25.852706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:25.853301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:25.853575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:25.853853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:25.854509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:25.854805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:25.856906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:25.862568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:25.864681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:25.869245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:25.874916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:25.971472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:42:25.977864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:25.981016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:26.083937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2112222-0c6e-4c72-b588-2be4943d32c0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:26.085378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d7b47-0a11-4618-bbb1-dc008668e39d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:26.122390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:26.122669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:26.123011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:26.123286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:26.123457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:26.123547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:26.123695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:26.123962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:26.124694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:26.126143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:26.127725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:25 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:26.133023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:26.142377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:26.143883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:26.152547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:42:26.163306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:26.164217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:42:26.165034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:42:26.165034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:42:26.165034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:26.165034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:26.170352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:26.226770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:26.242236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4292f6-3f6e-4361-8824-5949f7da80cd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:26.274660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28e078-4acf-47c0-af8e-8eeb5c4da009 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:26.276314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:26.276929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:26.277209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:26.277475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:26.278568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:26.280063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:26.281615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:26.288543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:26.308397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:26.309004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:26.309287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:26.309564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:26.310208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:26.310513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:26.311941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a603bb1-21cc-4fa1-b8ee-d8be57d54ebe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:26.330299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:26.385008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:26.410488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f39251-7d9e-4874-ba10-fe638ad3e9db None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:26.447991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:26.448610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:26.448914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:26.449184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:26.449922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.domain.id=default, user.id=654dd146616b4cbd8975ee15d286d26b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:26.450228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:26.451738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98921669-c4a0-4910-aab9-f9c78eadb0b6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:26.459800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Aug 9 17:42:26 2022] GET /identity/v3/role_assignments?user.id=654dd146616b4cbd8975ee15d286d26b&scope.domain.id=default&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 233 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:28.640794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d68d9-e6f8-41de-b048-f303fac6da79 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:28.641641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d68d9-e6f8-41de-b048-f303fac6da79 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:28.641641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d68d9-e6f8-41de-b048-f303fac6da79 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:28.642929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:28.648742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:28.649202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:28.649291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:28.684507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:28.686043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:28.713337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016a58-12bc-44fe-9ca9-20bfc4f98a94 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:28.713945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016a58-12bc-44fe-9ca9-20bfc4f98a94 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:28.714431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e016a58-12bc-44fe-9ca9-20bfc4f98a94 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:28.715992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 12/23] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:28.723238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:28.723873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:28.724397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:28.764098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:28.765643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:28.791834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 12/24] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:28.876453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:28.877300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:28.937076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:28.999631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:29.045660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:29.047397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a03cf9-125a-419c-aea6-aa75e43c9979 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:29.123370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:29.124012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.124291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.124564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.143589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:29.145124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:42:29.174397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:29.175384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:42:29.176745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:42:29.176745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:42:29.176745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:29.176745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 13/26] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:29.184114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:29.241056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed762a-21fe-458e-aee9-3a2266fa9673 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:29.340386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdfbe3-f93e-4877-b6a0-7801fec6dd9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:29.383710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:29.391734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:29.392392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.392668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.392936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.393606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:29.394218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.394548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.394823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.395507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:29.395805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:29.397182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:29.431169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 14/27] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:29.437334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:29.447966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:29.458845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:29.459777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:29.460533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:29.469177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 14/28] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 533 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:29.476002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:29.650168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab61a52-85c1-4ef6-b7cb-fefe624bc4f0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:29.693384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:29.694008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.694444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.694855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.695832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:29.698047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:29.793411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f31dad-de27-4d8e-b3ef-d3b11e32a475 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:29.803710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:29.840612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:29.841452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.841960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.842369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.843214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:29.843651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:29.845456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b96ad-c228-4080-89e8-e4e0b59184f3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:29.859753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 15/29] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:29.861379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbbf0-0727-49c7-8845-d2fc33260521 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.861799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbbf0-0727-49c7-8845-d2fc33260521 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.862294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbbf0-0727-49c7-8845-d2fc33260521 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.863977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 16/30] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:29.865320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b3e-7053-4583-be3a-2c6b7b84136d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.865788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b3e-7053-4583-be3a-2c6b7b84136d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.866226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b3e-7053-4583-be3a-2c6b7b84136d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.867912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 17/31] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:29.873160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 15/32] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:42:29 2022] POST /identity/v3/projects => generated 301 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:42:29.873160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:29.874787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a75d-cf04-47b2-9205-54bfb8efb618 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:29.875280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a75d-cf04-47b2-9205-54bfb8efb618 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:29.875719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0a75d-cf04-47b2-9205-54bfb8efb618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:29.898495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5fb0a75d-cf04-47b2-9205-54bfb8efb618 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:29.900126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fb0a75d-cf04-47b2-9205-54bfb8efb618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:29.960990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.011186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 18/33] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:30.013477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.013757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.014022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.048586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:30.049768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:30.078808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369093b9-ff38-4a58-8b37-78bec468e26f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.148321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 19/34] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:30.149156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.154954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.155625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.155908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.156186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.163867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.165674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.167786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 16/35] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:29 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:30.172815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.183795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.284559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b349285-c740-43ce-badf-8f85e73077d9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.328516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.329155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.329434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.329704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.330452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.332443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502377-754b-4c16-9677-dd24e17b1d93 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.338588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f016-51f3-4783-8975-0dc666da94c8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.347104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.381078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.381718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.382059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.382342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.383629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.385086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.391695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 17/36] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:30.391695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 20/37] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f166-861b-41d0-862f-ab6ba13b9073 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f166-861b-41d0-862f-ab6ba13b9073 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200f166-861b-41d0-862f-ab6ba13b9073 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:30.463020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f166-861b-41d0-862f-ab6ba13b9073 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.531837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af3f82-5cc3-4f34-85fd-b1227b778567 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.586839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.590542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.591224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.591637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.591914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.592611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.594051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda37d0-ff76-4b98-b880-c5481943dd81 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.607255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200f166-861b-41d0-862f-ab6ba13b9073 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.618340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 18/39] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/roles => generated 219 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:30.619877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d409ed-1d83-4f20-b07c-c57363e2c586 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.620163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d409ed-1d83-4f20-b07c-c57363e2c586 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.620433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d409ed-1d83-4f20-b07c-c57363e2c586 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.621886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 19/40] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:30.627010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.627316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.627556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.658246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:30.659846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:30.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.681360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] PATH_INFO: `/v3/domains/default/users/654dd146616b4cbd8975ee15d286d26b/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.684775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=654dd146616b4cbd8975ee15d286d26b, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.685985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af39c2-54de-461e-bebe-2bd58b6c0aa2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.707591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.714049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 22/41] 10.222.0.52 () {58 vars in 1361 bytes} [Tue Aug 9 17:42:30 2022] PUT /identity/v3/domains/default/users/654dd146616b4cbd8975ee15d286d26b/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:42:30.715368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.715368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.715739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.742244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:30.743982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:30.767047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 20/42] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:30.790169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.838450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 23/43] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:30.864975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:30.931004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e855c3-e351-4a7d-b1e7-623009d23310 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.959242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:30.974631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:30.975736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:30.976029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:30.976468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:30.977148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:30.978578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:30.988134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554c733-f5ae-48bf-b383-78fb21aa9aab None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:30.994798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 21/44] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 9 17:42:30 2022] POST /identity/v3/roles => generated 223 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:42:31.032299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:31.032952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:31.033239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:31.033512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:31.034676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:31.036725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c78ae6-9623-4410-b787-45d8f92c0c4f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:31.038527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 24/45] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:31.045362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:31.080596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:31.180215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14465eb5-4c9e-40e7-a179-689f6b5e65e7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:31.217131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:31.217768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:31.218120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:31.218465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:31.219171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:31.220653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb4e16-8f0b-439e-acba-8f7eefa54fc5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:31.255914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:42:31 2022] POST /identity/v3/projects => generated 312 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:42:32.761026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62774539-9b05-4740-b565-2fd69b85a1bb None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:32.761575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62774539-9b05-4740-b565-2fd69b85a1bb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:32.761575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62774539-9b05-4740-b565-2fd69b85a1bb None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:32.763074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:32.768282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:32.768935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:32.768935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:32.789370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:32.790579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:32.867143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 23/48] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:32.891977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62774539-9b05-4740-b565-2fd69b85a1bb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.015976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62774539-9b05-4740-b565-2fd69b85a1bb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.068393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.069036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.069321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.069924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.080298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:33.085609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:33.086441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:33.087549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 26/49] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:33.092900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.101740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:33.201834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2edb0b2-5e09-4ce0-a667-5f03e37a6128 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.236360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.237005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.237291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.237568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.238239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:33.238544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.239963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac28b-d3e9-47c0-a71e-0a2a4193e8c5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.256045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 24/50] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:33.257950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.278604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:33.290084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4004e-5e08-46ca-a896-e5ff05d96314 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.290376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4004e-5e08-46ca-a896-e5ff05d96314 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.290653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4004e-5e08-46ca-a896-e5ff05d96314 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.292028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 25/51] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:33.298084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.298369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.298643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.317595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:33.318786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:33.382153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab2ecf-a522-4add-a8ca-1b76f63c9b7d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.383264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:33.389611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 26/52] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:33.413316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.417253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.418228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.418581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.418914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.420217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.421756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.423770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:33 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:33.432375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.532415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:33.541182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fe085-aebf-4af5-83e4-a6ff2da147db None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.560251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92f60d-2589-4dc0-9ec9-3f355fe311d9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.587013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.587800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.588182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.588539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.589777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.591491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.593500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 27/54] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:33.599531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.599878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.600588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.601076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.601616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.603773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.605274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.609242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:33.617890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:33.676564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:33.733868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcd571-125a-433a-ac5d-949e98440c67 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.736382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e50a6-e848-440a-a552-07e79f26c410 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:33.775165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.775845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.776070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.776344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.777078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.domain.id=default, user.id=654dd146616b4cbd8975ee15d286d26b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:33.777210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:33.777583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.777814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:33.778094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:33.778368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:33.778933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17eac5e-22c0-48fc-a512-f040101a0adb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.779207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:33.780513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cdcbc-d759-4132-a8e0-5e4c2fe2dc95 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:33.787991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Aug 9 17:42:33 2022] GET /identity/v3/role_assignments?user.id=654dd146616b4cbd8975ee15d286d26b&scope.domain.id=default&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:33.810812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 28/57] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Aug 9 17:42:33 2022] POST /identity/v3/projects => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:42:35.870304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6917c9-7929-4d5b-a6d6-71b4186e7e9b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:35.870829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6917c9-7929-4d5b-a6d6-71b4186e7e9b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:35.870915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6917c9-7929-4d5b-a6d6-71b4186e7e9b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:35.872252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:35.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:35.878071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:35.878194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:35.896726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:35.897935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:35.967157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:35.991917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6917c9-7929-4d5b-a6d6-71b4186e7e9b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:36.001105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:36.094869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6917c9-7929-4d5b-a6d6-71b4186e7e9b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:36.128581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:36.129136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:36.129414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:36.129687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:36.130784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:36.132250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0085e33-3cc1-45a4-8bcc-cc3a7edc8884 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:36.133967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:36.139773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:36.189642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:36.241033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007531ef-c274-465b-803e-d20bb2df6c2a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:36.273706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:36.274305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:36.274585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:36.274859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:36.275533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:36.277058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fcd0a-a4a6-4151-8cfc-cefedbbf3159 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:36.303669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:42:36 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:39.639448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f18c2-b125-410f-8083-3ec7e590ae8f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:39.640237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f18c2-b125-410f-8083-3ec7e590ae8f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:39.640237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f18c2-b125-410f-8083-3ec7e590ae8f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:39.641734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:39.648829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:39.649647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:39.650091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:39.672302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:39.673565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:39.714578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0767-1e77-4a82-8f60-cb5378b65882 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:39.714868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0767-1e77-4a82-8f60-cb5378b65882 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:39.715374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0767-1e77-4a82-8f60-cb5378b65882 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:39.717545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 33/63] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:39.722548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:39.723098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:39.723598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:39.762939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:39.764507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:39.814950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 31/64] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:39.840321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:39.924985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:39.926142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 34/65] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:39.961763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:39.971335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.007060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d823d38-9316-4437-be20-8f4ae2620856 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.043409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.044445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.044896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.045392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.071039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5e423720-796a-423d-8ebf-a0d98156d96c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:40.076279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:40.077223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:40.078096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:40.080096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 32/66] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:40.081958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e679-c5a5-43bf-9374-3ee12e259199 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.082392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e679-c5a5-43bf-9374-3ee12e259199 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.082829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e679-c5a5-43bf-9374-3ee12e259199 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.084527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 33/67] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:40.093020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e679-c5a5-43bf-9374-3ee12e259199 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.096533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca47ca-512d-41a7-8f11-b11584206060 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.139428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.140069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.140355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.140633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.153876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8e6a9471-3648-418b-aa58-7878e229ea79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:40.158596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:40.160318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:40.161278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:40.168042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 35/68] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:40.168575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431d29-1e63-47a6-b9c1-0b36a08009b4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.168862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431d29-1e63-47a6-b9c1-0b36a08009b4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.169652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431d29-1e63-47a6-b9c1-0b36a08009b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.191373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-12431d29-1e63-47a6-b9c1-0b36a08009b4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:40.192333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-12431d29-1e63-47a6-b9c1-0b36a08009b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:40.208994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.218814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e679-c5a5-43bf-9374-3ee12e259199 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.254649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.258912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.258912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.259182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.259436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.260236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:40.260541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:40.261937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08215255-5e9c-431b-9198-4310e9de7aea None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:40.267268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 36/69] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:40.274821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaddb4e-4f07-47cb-b941-6547cf6b431c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.274821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaddb4e-4f07-47cb-b941-6547cf6b431c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.274821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaddb4e-4f07-47cb-b941-6547cf6b431c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.274821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 37/70] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:40.274821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaddb4e-4f07-47cb-b941-6547cf6b431c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.280021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 34/71] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:40.281199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.281557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.281901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.301992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:40.303213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:40.389487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 35/72] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:40.390347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.411704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaddb4e-4f07-47cb-b941-6547cf6b431c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.469911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.470815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.471276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.471736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.473333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:40.473849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:40.476276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:40.489877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 38/73] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:40.491131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.513928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.532620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9a9ae-2895-4d25-bb1e-5c5d147605a9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.550256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.567622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.568255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.568537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.568813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.583539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:40.585007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:40.586642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 36/74] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:40.587438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.604458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff961d35-38a4-428f-8e72-4d4408b105aa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.642131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.642796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.643086 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.643368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.655022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-dda98afb-6158-4ed8-a9dc-b9f8d0cf9cdf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:40.659804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:40.660671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:40.661428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 39/75] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:40.662272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26fa3a-782b-4d12-8e9e-e13473f7fce1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.662547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26fa3a-782b-4d12-8e9e-e13473f7fce1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.662788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26fa3a-782b-4d12-8e9e-e13473f7fce1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.664023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 40/76] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:40.664721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26fa3a-782b-4d12-8e9e-e13473f7fce1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.707353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064966d-3038-4c94-9174-e24f3707a58b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.748199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.748968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.748968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.749893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.761264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:40.765951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:40.767053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:40.767849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.781765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26fa3a-782b-4d12-8e9e-e13473f7fce1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.781913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.798887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:40.827548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.828188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.828472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.828748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.829949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:40.831541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:40.833662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:40.834482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:40.898001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243176fd-71e8-4250-9d35-e6a9e2df8114 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.934522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.935384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.935775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.936185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.943342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01652a1a-45ba-44e2-a606-5083b7cb98b8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:40.987031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:40.988140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:40.988423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:40.988696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:40.989359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:40.989657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:40.991067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ce561-f2f0-40a6-a059-3db0247d78d3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:40.999721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 42/79] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.000809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.001091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.001370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.017370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.018856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.019962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:41.021156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:41.021695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.075871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.105465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 43/80] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:41.106595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.143446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 38/81] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:41.144629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.221894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807f1d3-5d13-4d7a-976f-9e2148403cae None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.269348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f34f66-080d-49a2-a2f4-f86c9ff8a66f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.272144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.272773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.273001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.273836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.274288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.275451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:41.275949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.278264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.292617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 44/82] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.293756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.320927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.321544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.321829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.322099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.342465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.401030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.402638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.411472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7d1c6-3595-46ed-9230-a50790e14e10 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.453816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.454466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.454746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.455046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.456201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.457708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.459445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 45/83] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.460219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.541873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 39/84] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:41.542657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.549666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.573206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6c77e-91ad-4824-af1c-dc8a51c05742 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.580687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.612319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.613758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.613758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.614028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.615282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.616825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.625265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 46/85] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.625265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.661271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d9f03-6f55-4b42-8b9f-ccc55b19a5f2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.701353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.701982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.702262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.702532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.703758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.705248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.706916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 40/86] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.707699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.749369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f6b0f-b6fc-4bf9-ad17-f47b7db3abe2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.797693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.799212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.799957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.800348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.800626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.801792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.803280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.811183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 47/87] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.811183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:41.838099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c42fc-f61f-419a-8615-c9677344268b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.853154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:41.876941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.877734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.877927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.878499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.920686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abd72-5897-4970-917d-ff3c452f3119 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:41.954510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:41.955219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:41.955501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:41.955739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.955836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:41.957511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:41.957746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.959026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:41.960684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 48/88] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:41.961421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.045256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.079528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 41/89] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:42.080388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.088630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42d44e-c0dd-4820-bc46-8a858409d543 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:42.100874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.134338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:42.134960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:42.135276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:42.135551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:42.136853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:42.137658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:42.171863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 49/90] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:42:41 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:42.173122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.200616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aee2d4-48c1-4344-9ca9-807afe40dfc0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:42.250509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:42.251188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:42.251560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:42.251839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:42.252612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:42.252936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:42.254953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:42.279168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 42/91] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:42.279168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.337744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.371002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a58cf7-6edb-4629-8a05-1d61d2e8fb01 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:42.430036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:42.430681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:42.430966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:42.431327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:42.465403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9ddb9-347b-48bd-9c27-fe08174555c1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:42.486420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.521282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:42.521913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:42.522580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:42.522691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:42.523457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=654dd146616b4cbd8975ee15d286d26b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:42.524172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:42.525815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:42.534880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 43/92] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/role_assignments?user.id=654dd146616b4cbd8975ee15d286d26b&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:42.537445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.541937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:42.548167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:42.744704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.748773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9694921-867b-4779-b023-244d2a174ba5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:42.758588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 50/93] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:42.768063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.816840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:42.817571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:42.818319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:42.818618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:42.820387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:42.821880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:42.824008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 44/94] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:42.832409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:42.877581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:42.971544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516583a-c28d-4388-b162-2f2026a2b59b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:43.037227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:43.038138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:43.038582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:43.039046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] PATH_INFO: `/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:43.044502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7c1bdda0f040f4be95392dd75f4d16)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:43.045984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:43.048131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 51/95] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:43.055154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:43.083924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df97aa6-ef5f-4637-862d-5597ab23a404 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:43.130307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:43.130946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:43.135140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:43.135140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:43.135140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:43.135140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:43.135140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a6e1-2f37-4ea9-a1d7-2c88e6853b34 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:43.140196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 45/96] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:42 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=da668845afcd4a95bf6de91627f699b4 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:43.227684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b3604-05c6-448c-bb63-31e39006dbb7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:43.268382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:43.269789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:43.270216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:43.270628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:43.271600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=3d7c1bdda0f040f4be95392dd75f4d16, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:43.272068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:43.273745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb713c31-cfe0-4139-91da-6e4f70ec0b96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:43.276190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:43.280767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 52/97] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:43 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=3d7c1bdda0f040f4be95392dd75f4d16&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:45.424253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363150bd-671d-456e-9f5d-f944466cfa0a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.424769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363150bd-671d-456e-9f5d-f944466cfa0a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.424842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363150bd-671d-456e-9f5d-f944466cfa0a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.426362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 46/98] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:45.441266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e1c7-9dd8-4d5b-bcf2-e0d44e87e9ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.442237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e1c7-9dd8-4d5b-bcf2-e0d44e87e9ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.442878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e1c7-9dd8-4d5b-bcf2-e0d44e87e9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.472037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0d65e1c7-9dd8-4d5b-bcf2-e0d44e87e9ac None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:45.473613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d65e1c7-9dd8-4d5b-bcf2-e0d44e87e9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:45.560227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 53/99] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:45.595837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363150bd-671d-456e-9f5d-f944466cfa0a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:45.607772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848cf3-7b0b-4ebf-8e8d-f28d47c1d010 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.608126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848cf3-7b0b-4ebf-8e8d-f28d47c1d010 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.608491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848cf3-7b0b-4ebf-8e8d-f28d47c1d010 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.609870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 54/100] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:45.627106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058ddc-ae2a-4535-a5af-d5aaffe107e1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.627434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058ddc-ae2a-4535-a5af-d5aaffe107e1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.627717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058ddc-ae2a-4535-a5af-d5aaffe107e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.650352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:45.662766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1c058ddc-ae2a-4535-a5af-d5aaffe107e1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:45.663981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1c058ddc-ae2a-4535-a5af-d5aaffe107e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:45.744477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363150bd-671d-456e-9f5d-f944466cfa0a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:45.779805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:45.780442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.780717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.780989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.794908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-71fea01a-6bae-42d1-836b-56a398cb9438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:45.800575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:45.801437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:45.803634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 47/101] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:45.809537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d6e9-1181-475e-903b-f49b826911c6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.809833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d6e9-1181-475e-903b-f49b826911c6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.810118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d6e9-1181-475e-903b-f49b826911c6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.811695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 48/102] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:45.812372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5d6e9-1181-475e-903b-f49b826911c6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:45.825568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 55/103] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:45.826674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.826932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.827232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.861013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:45.862204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:45.925960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5d6e9-1181-475e-903b-f49b826911c6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:45.927847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:45.968838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:45.969622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:45.969901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:45.970170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:45.971349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:45.971654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:45.973580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:45.984350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 49/104] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:45.985185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:45.985296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.019726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 56/105] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:46.021883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.116854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1903d3-d95b-441a-b421-f54c309d1137 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.157823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304d625-f8d8-4fc9-8c41-56d044d02586 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.158016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.158598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.158898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.159195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.174002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8c761f12-0595-4fd3-ae50-7ded88c7ed22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:46.180530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:46.181465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:46.182321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 50/106] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:46.183571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a6de-742a-48c0-b706-84b1e4847faf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.183571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a6de-742a-48c0-b706-84b1e4847faf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.183571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a6de-742a-48c0-b706-84b1e4847faf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.184911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 51/107] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:46.185658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4a6de-742a-48c0-b706-84b1e4847faf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.206252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.210342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.211081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.211447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.211550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] PATH_INFO: `/v3/users/b369254e9f9141b58cefeaa8ca19c0b5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.227533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] RBAC: Authorizing `identity:get_user(user_id=b369254e9f9141b58cefeaa8ca19c0b5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.228996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c10346-05d2-498b-9994-77feeeee629c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.230913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 57/108] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/users/b369254e9f9141b58cefeaa8ca19c0b5 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:46.232242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81792e8-a26c-4848-9869-4d4d4869fc71 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.232660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81792e8-a26c-4848-9869-4d4d4869fc71 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.232949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81792e8-a26c-4848-9869-4d4d4869fc71 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.234347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 58/109] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:46.236149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81792e8-a26c-4848-9869-4d4d4869fc71 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.293806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.316568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4a6de-742a-48c0-b706-84b1e4847faf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.363500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.364641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.364975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.365315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.366458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.367453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81792e8-a26c-4848-9869-4d4d4869fc71 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.367963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc365438-b53f-43e8-81fe-2f2805d7c7e8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.369749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 52/110] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:46.376387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7e2d-b511-4b75-89c5-cb2dc1a15ad2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.376681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7e2d-b511-4b75-89c5-cb2dc1a15ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.376960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f7e2d-b511-4b75-89c5-cb2dc1a15ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.401154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7a1f7e2d-b511-4b75-89c5-cb2dc1a15ad2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:46.403427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a1f7e2d-b511-4b75-89c5-cb2dc1a15ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:46.407796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.408417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.408702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.408983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.410217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:46.410521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.411973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.421024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 59/111] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:46.421843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.507010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 53/112] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:46.507010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.507010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.507010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.521542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:46.522945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:46.531714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.552060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba23954-d1a5-49bf-8be7-89f508954aa8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.561613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.587584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.588268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.588596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.588824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.598729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 54/113] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:46.607180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.681760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.683509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.712237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b0b80-087d-482f-b01e-97b1d372c1fa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.751198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.751816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.752097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.752405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.753014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.755025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.772475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.794896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 55/114] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:42:46 2022] POST /identity/v3/projects => generated 302 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:46.794896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.819941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 60/115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:46.821049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:46.836845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:46.918577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7423-9068-4dff-921e-c276f690b799 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.956110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05557f-f31e-408e-83c0-872962b59817 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:46.959601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:46.960242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.961485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.961485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.961950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:46.963519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af614d-638e-444f-adcf-f4a3a9faa499 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:46.965373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 56/116] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:46.966419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089104c-4e67-498d-8ecf-10d095f62066 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:46.966711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089104c-4e67-498d-8ecf-10d095f62066 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:46.967071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089104c-4e67-498d-8ecf-10d095f62066 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:46.968479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 57/117] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:46.969115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089104c-4e67-498d-8ecf-10d095f62066 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.006479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.007189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.007450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.007724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.025583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:47.031400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:47.032524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 61/118] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:47.033460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.065538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.120377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.127339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089104c-4e67-498d-8ecf-10d095f62066 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.167408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e49ef-3025-4071-aed2-32cafaf0cc1a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.171883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.172644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.173094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.173524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.187336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:47.192870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:47.193933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:47.194880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:47.195641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 58/119] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:47.196319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.215674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.216287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.216770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.216899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.217970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.219465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62a9c-9b61-4838-a498-56aae32766d8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.226757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 62/120] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:47.226757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.226757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.226757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.245728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:47.247321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:47.318426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aba1dd-9732-414c-9e64-65892f05d682 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.324773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 63/121] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:47.325575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.352474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.367604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.368788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.369193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.369561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.372141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.376081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.382828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 59/122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:47.382828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.394252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.429909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9258bd-afb1-414f-95dd-be6aaa0eb8e9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.463571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.464170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.464451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.464719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.465366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:47.465663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.467052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.475840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 64/123] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:47.481402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.501043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7f199-cc68-435b-9c68-828b247a10a7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.535463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.536246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.536535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.536869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.537539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:47.537846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.539268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.552566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 60/124] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:47.552566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.599250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ceed0c-5830-489a-9ccd-5098776cadf1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.610790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.628877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.638793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.639486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.639770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.640421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.641131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=b369254e9f9141b58cefeaa8ca19c0b5) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:47.641742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.643488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.655168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 65/125] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/role_assignments?user.id=b369254e9f9141b58cefeaa8ca19c0b5&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:47.657001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.674807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f186b1-95ca-4d4f-9576-4aab7754cc2d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.719242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.721421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.722248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.722552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.745381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:47.751624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:47.752647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:47.753461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:47.755867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 61/126] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:47.756962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.827500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250606d-337e-404c-a5b6-71b27c71f418 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.869471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.870034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.870312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.870586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/users/d0ee218e69104872aa22674813a5ef39/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.872538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, user_id=d0ee218e69104872aa22674813a5ef39, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.874187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:47.892713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45d5b-9998-47fc-9b71-79387368824c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:47.899102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 66/127] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:42:47 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/users/d0ee218e69104872aa22674813a5ef39/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:42:47.902479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:47.928566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:47.984818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:47.985919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:47.988899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:47.989501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:47.991205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:47.994366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.002070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 62/128] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.004459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.037932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:48.149047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3673ed-002d-4451-ab52-1d882c16fad8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.154638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86e03a-0c55-472e-bdf1-fbe8539d706f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.192973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.193682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.193899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.194174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.195516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.197801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.199392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 67/129] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:47 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.200076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.205202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.206222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.207228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.207742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.208628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:48.209133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.211390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.222176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 63/130] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.224082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.288377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:48.356348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe87ca4-70a1-4a5b-a95a-82ddaa544314 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.404449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45538619-51a1-4657-9146-132315a79f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.405736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45538619-51a1-4657-9146-132315a79f8a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.406212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45538619-51a1-4657-9146-132315a79f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.406683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45538619-51a1-4657-9146-132315a79f8a None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.408276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45538619-51a1-4657-9146-132315a79f8a None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.410303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45538619-51a1-4657-9146-132315a79f8a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.412969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 64/131] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:48 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.415076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45538619-51a1-4657-9146-132315a79f8a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.421026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c05769-ff3d-4bbb-82e3-8cd48a70f72f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.474824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:48.541307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45538619-51a1-4657-9146-132315a79f8a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.544963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.544963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.545272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.545568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.546721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.548309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.550522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 68/132] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:48 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.551311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.558466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:48.583773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.584820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.585303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.585767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.587466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.589957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.592111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 65/133] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:48 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:48.593267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.711114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e47bce-1f4d-4103-b157-4a5280e8796a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.746349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.747331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.747609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/users/d0ee218e69104872aa22674813a5ef39/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.749866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, user_id=d0ee218e69104872aa22674813a5ef39, role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.751376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.768096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 66/134] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:42:48 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/users/d0ee218e69104872aa22674813a5ef39/roles/da668845afcd4a95bf6de91627f699b4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:42:48.780433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:48.840995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe5fc0-f942-4508-a336-691c3abf1d6c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:48.859454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:48.932164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394580-da4a-458b-881c-1e222854343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:48.932881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394580-da4a-458b-881c-1e222854343b None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:48.933618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394580-da4a-458b-881c-1e222854343b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:48.933910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c394580-da4a-458b-881c-1e222854343b None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/users/654dd146616b4cbd8975ee15d286d26b/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:48.943832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c394580-da4a-458b-881c-1e222854343b None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, user_id=654dd146616b4cbd8975ee15d286d26b, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:48.946156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c394580-da4a-458b-881c-1e222854343b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:48.975207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 69/135] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:42:48 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/users/654dd146616b4cbd8975ee15d286d26b/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:42:49.008319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32338f-42e3-4cbe-9d09-7d2d4e1f7f1a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:49.051553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:49.052212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.052497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.052830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] PATH_INFO: `/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.054075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7c1bdda0f040f4be95392dd75f4d16)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:49.055684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92955c-8096-4d58-bf3c-5cb6f88bfe19 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:49.057963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 67/136] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:48 2022] GET /identity/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:49.064121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394580-da4a-458b-881c-1e222854343b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:49.153184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:49.177507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaeb5-c455-4e9c-bb4d-98a3094d34bf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.177804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaeb5-c455-4e9c-bb4d-98a3094d34bf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.178075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaeb5-c455-4e9c-bb4d-98a3094d34bf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.180338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 68/137] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:49.186413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.186705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.186996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.211438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:49.213207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:49.238839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c394580-da4a-458b-881c-1e222854343b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:49.300527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:49.301357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.301641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.301914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] PATH_INFO: `/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16/users/d0ee218e69104872aa22674813a5ef39/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.307106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 69/138] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:49.308468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d7c1bdda0f040f4be95392dd75f4d16, user_id=d0ee218e69104872aa22674813a5ef39, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:49.310022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:49.330597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:49.331590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 70/139] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:42:49 2022] PUT /identity/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16/users/d0ee218e69104872aa22674813a5ef39/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:42:49.375486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:49.455413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563c33e-de06-4216-bb52-edfa6c6c7b11 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:49.495023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:49.495587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.495870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.496146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.497344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:49.498843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2ff1c-ac91-4620-bc63-82e46a4a5ca2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:49.500755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 70/140] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:49.508171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:49.639081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4305e4-4001-4d53-bfea-3d7cd1925681 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:49.649245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:49.683693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:49.684316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:49.684596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:49.684869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:49.685521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:49.687054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a2394-ff44-414e-9bf7-acaecd1b199f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:49.712596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 71/141] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:42:49 2022] POST /identity/v3/users => generated 283 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:50.259359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092181-4083-461c-ad3b-0de70febf85e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.260406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092181-4083-461c-ad3b-0de70febf85e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.260895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092181-4083-461c-ad3b-0de70febf85e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.262483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 71/142] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:50.268322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.268814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.268913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.294739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:50.296019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:50.386823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 72/143] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:50.420300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b092181-4083-461c-ad3b-0de70febf85e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:50.558295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b092181-4083-461c-ad3b-0de70febf85e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:50.561909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:50.605426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:50.606065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.606603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.606915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.620857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-55938269-2c15-4aa1-ba75-8b691c20d5d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:50.625655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:50.626485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:50.627258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 72/144] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:50.634248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:50.759443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:50.766010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a612e5-cf6a-4aa9-b9a6-fcf6f9837669 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:50.804618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:50.805246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.805577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.806093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.806776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:50.807332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:50.808803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:50.810182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37fd8f-7cba-4b92-95ee-f780bf73627f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.810474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37fd8f-7cba-4b92-95ee-f780bf73627f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.810754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37fd8f-7cba-4b92-95ee-f780bf73627f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.812129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 73/145] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:50.818154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 73/146] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:50.819698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:50.819974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:50.820230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:50.825167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:50.857303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:50.858514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:50.988858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb07313-0c90-42ca-9dfa-4e7adbb36ebc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.012870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 74/147] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:51.050241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.052020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.052893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.053175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.053450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] PATH_INFO: `/v3/users/b369254e9f9141b58cefeaa8ca19c0b5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.055903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] RBAC: Authorizing `identity:get_user(user_id=b369254e9f9141b58cefeaa8ca19c0b5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:51.058731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:51.060652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 74/148] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:50 2022] GET /identity/v3/users/b369254e9f9141b58cefeaa8ca19c0b5 => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:51.068824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.119826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:51.267120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cbf9-d6c3-4c32-9d68-9007b63d1a5b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.312766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8b07a-5442-4ad7-9b84-c01b3642a3df None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.316710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:51.350382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.351275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.351412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.351630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.355702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:51.357821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:51.360505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 75/149] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:51 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:51.365986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.373638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.374490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.374933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.375411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.392756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:51.398931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:51.400046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:51.400972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:51.403973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 75/150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:51.429231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.507249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:51.519308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4666-c301-4502-b3e3-c743659a6bfa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.575742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.576432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.576712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.576993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/users/b369254e9f9141b58cefeaa8ca19c0b5/roles/5e4ea42893434cc9b1df29922b3cab0d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.579053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, user_id=b369254e9f9141b58cefeaa8ca19c0b5, role_id=5e4ea42893434cc9b1df29922b3cab0d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:51.580502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:51.605008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 76/151] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:42:51 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/users/b369254e9f9141b58cefeaa8ca19c0b5/roles/5e4ea42893434cc9b1df29922b3cab0d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:42:51.624538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2127e-c21d-4087-b3ec-2df35e3981d8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.677363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.678401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.678920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.679398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.680274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:51.680862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:51.682446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:51.697614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 76/152] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:51.703463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.879418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332eadf-86f9-4b0c-b747-7a716b36f9e1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:51.919897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:51.920533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:51.920813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:51.921084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:51.927862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:51.929371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:51.931110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 77/153] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:51 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:51.937036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:51.966121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:52.093180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88fdfb0-f64e-4e58-996e-e9021d7236a4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:52.135647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:52.136180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.136451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.136719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.224353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:52.226186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:52.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:52.348859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 77/154] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:52.354519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:52.397572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:52.507981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092b61f-90e9-46dd-bbec-ac69f9a8911d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:52.555698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:52.556339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.556618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.556893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.558090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:52.559579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8e52e-3bbd-449d-abdd-96ca6b7594d1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:52.561461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 78/155] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:52 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:52.567986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:52.710006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1a7f-207f-4cb2-930e-5c840808955a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:52.734815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817065b4-358a-430a-b19b-2e64b4a3fc08 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.736234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817065b4-358a-430a-b19b-2e64b4a3fc08 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.736687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817065b4-358a-430a-b19b-2e64b4a3fc08 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.740191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 79/156] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:52.749466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:52.764497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.764981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.766010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.801127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:52.802140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.802615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.803108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.804352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:52.804847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:52.807562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f3519-15cf-4f66-9073-8000a1d30ed2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:52.817067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:52.820222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:52.824069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 78/157] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:52 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:52.927367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a99db-c050-47a8-9809-41d96b6b9e08 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.927871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a99db-c050-47a8-9809-41d96b6b9e08 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.928643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a99db-c050-47a8-9809-41d96b6b9e08 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.931807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 79/158] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:52.941445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:52.942373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:52.943128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:52.967763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 80/159] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:53.005560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:53.008154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:53.042730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:53.096323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:53.212498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 80/160] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:53.278933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d276a5-4104-419e-a478-2e16e928b7ca None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:53.282759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:53.364293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:53.365705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:53.366187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:53.366657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:53.391386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:53.396207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:53.397043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:53.398207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 81/161] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:42:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:42:53.407851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:53.545681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:53.614657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc46ad-898d-49aa-9e9f-e0c600ccc924 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:53.615190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ad4-9203-42f4-800c-825f4186d0a5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:53.646542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:53.709478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:53.710481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:53.710963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:53.715500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:53.717165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:53.717824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:53.719949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d2a5e-3e68-4f0d-bccb-23a86e521bca None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:53.736059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 82/162] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:42:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:53.738662 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:53.738856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd3b5-0730-4fdf-9df0-3a723ab389c1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:53.739139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd3b5-0730-4fdf-9df0-3a723ab389c1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:53.739581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd3b5-0730-4fdf-9df0-3a723ab389c1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:53.739753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:53.740486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:53.740925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:53.747312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 83/163] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:53.756244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:53.756719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:53.757181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:53.785472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:53.795034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:53.796000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:53.796823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:53.798021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 81/164] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 516 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:53.799279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:53.803343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:53.805490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:53.981892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 84/165] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 17:42:53.982753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.018483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037db20-919a-4690-91e0-a469ab1db402 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.071084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.072197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.072627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.073018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.077781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:54.079517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:54.081306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 82/166] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:53 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:54.082280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.159973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:54.220957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:54.272940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323afac2-8363-41bf-8c2e-a727b8c1fc2f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.307810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecad75e-cfe6-40fb-ac63-bb082e062392 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.339980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.340530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.340806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.341071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.341760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:54.342059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:54.343540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:54.357461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 85/167] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:54.358621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.383470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.384138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.384423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.384697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.410580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:54.416190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:54.417041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:54.417892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:54.423117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 83/168] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:54.424159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.581714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60516e6-a94a-479a-95ba-167d13054468 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.629080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d825155-930a-435b-a6b8-17ec3da74641 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.632574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:54.663009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:54.670870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.672939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.673225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.673498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.696819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.697445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.697715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.697980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] PATH_INFO: `/v3/users/654dd146616b4cbd8975ee15d286d26b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.699128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] RBAC: Authorizing `identity:get_user(user_id=654dd146616b4cbd8975ee15d286d26b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:54.700569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:54.702472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 84/169] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:54 2022] GET /identity/v3/users/654dd146616b4cbd8975ee15d286d26b => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:54.703332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.810477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536b385-9eee-4647-83cc-09e2601c6221 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:54.840053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:54.841869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec8ab-cbf4-4e42-a325-da463a9662e3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:54.856792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:54.857422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.857703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.857978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.858661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:54.858962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:54.860522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87755d0-c0b1-40b6-a4ec-472fd700f1d1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:54.879896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 85/170] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:54.881084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c8d1-ff6f-4290-a3c2-c528f6b8603a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.881368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c8d1-ff6f-4290-a3c2-c528f6b8603a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.881635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776c8d1-ff6f-4290-a3c2-c528f6b8603a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:54.883058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 86/171] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:54.883576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776c8d1-ff6f-4290-a3c2-c528f6b8603a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:54.919860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:54.975037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 86/172] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:54.976096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:54.976382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:54.976657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.005457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:55.006936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:55.044369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776c8d1-ff6f-4290-a3c2-c528f6b8603a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.046648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.100100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 87/173] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:55.101810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.103160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.103904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.104284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.104660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.179789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.181325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0dbcc-dc6e-452c-a94f-6f7c1605720e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.210687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.229851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b74b6-a36b-4b34-9514-869050e8ef69 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.269000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.269719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.269922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.270197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] PATH_INFO: `/v3/users/d0ee218e69104872aa22674813a5ef39` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.271364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ee218e69104872aa22674813a5ef39)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.272885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.279224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 88/174] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/users/d0ee218e69104872aa22674813a5ef39 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:55.279224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.297755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 87/175] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:55.298935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a5bd-ed17-4bae-bd3b-e68416c5709c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.299318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a5bd-ed17-4bae-bd3b-e68416c5709c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.299599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5a5bd-ed17-4bae-bd3b-e68416c5709c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.306086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 88/176] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:55.306086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5a5bd-ed17-4bae-bd3b-e68416c5709c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.312625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.391876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bc721-82be-43c8-a487-a2b41d2cf203 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.414621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5a5bd-ed17-4bae-bd3b-e68416c5709c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.431952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.432505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.432890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.433164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.434401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.435878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afee29d-46ff-4cee-9dba-5b48c2dc02a3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.437687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 89/177] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:55.443397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49771-f9b1-4735-bb9b-beb4eb0af06c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.443397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49771-f9b1-4735-bb9b-beb4eb0af06c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.443397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49771-f9b1-4735-bb9b-beb4eb0af06c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.443397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 90/178] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:55.443397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b49771-f9b1-4735-bb9b-beb4eb0af06c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.450102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.459263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.459977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.460249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.460424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.472669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:55.477872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:55.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:55.479951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 89/179] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:55.480803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.567557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.572532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b49771-f9b1-4735-bb9b-beb4eb0af06c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.593684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef863ef-1ea2-4d28-a14d-3738682070f2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.609327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.609932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.610235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.610474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.629220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.629854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.630134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.630406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.631976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.633437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71b098-2b1f-4ec5-98c1-db62c7dd5574 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.635086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 90/180] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:55.635086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.635086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.635086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.655693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:55.656917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:55.687721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.687721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ea7f6-019c-481a-b0ba-b553cb5b5999 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.724777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.743267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 91/181] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:55.744088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.793630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:55.801877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 91/182] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:55.803437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.803913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.804026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.822929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:55.824159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:55.861839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843142c5-acc2-488d-98f7-1b0df8d611cf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:55.892632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 92/183] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:55.893558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.898674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:55.899328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.899611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.899884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.900630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:55.900937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:55.902865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29fa6f3-a5b9-44a0-8a51-c1d155f20d9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 92/184] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=da668845afcd4a95bf6de91627f699b4 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d6378-c404-44e2-b8e5-350782009221 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d6378-c404-44e2-b8e5-350782009221 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d6378-c404-44e2-b8e5-350782009221 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 93/185] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:55.915870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d6378-c404-44e2-b8e5-350782009221 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:55.983798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.023782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad8f05-8cec-451a-9bef-be5a5aa4b808 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.045801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d6378-c404-44e2-b8e5-350782009221 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.054628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.073120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.073737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.074010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.074281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.075132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:56.075332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.076663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.086148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 93/186] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.087045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.089717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.090376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.090649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.090960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.091834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=654dd146616b4cbd8975ee15d286d26b) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:56.092146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.093665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.101492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:55 2022] GET /identity/v3/role_assignments?user.id=654dd146616b4cbd8975ee15d286d26b&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.102396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.233173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc60fcb-0813-4ded-9366-7b7adf518d10 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.255829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231384cc-9aa1-4837-9387-666fd97a4434 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.271896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.272793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.273331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.273627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.273905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.275492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.277226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc153f-2f1c-4e95-99ff-8bf2c8e6991c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.279043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 94/188] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.285218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.285497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.285766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.303038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.303686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.304030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.304329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] PATH_INFO: `/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.305474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7c1bdda0f040f4be95392dd75f4d16)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.307932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:56.308055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.308129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:56.308774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 95/189] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/projects/3d7c1bdda0f040f4be95392dd75f4d16 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.309610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.381612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.397571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 95/190] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:56.398505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.434884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15a50-97f2-40d8-b0f2-8ea29fb92ea7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.477890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.478613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.479081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.479517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.494899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:56.504582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:56.505487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:42:56.506313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:56.506917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 96/191] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:56.507612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.528128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc180ed7-ca52-41fb-aa0a-878c147a1d43 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.556259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.564503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.565132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.565942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.566214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.579576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.581666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.583354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 96/192] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.583354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.625711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdf20-049a-4750-bf60-27343cdb938c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.636454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.663288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.663895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.664171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.664595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.665374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.666869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.690723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 97/193] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:42:56 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:56.691578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.706466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b254b3-6188-4398-b187-dac6cc711a34 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.761307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.761841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.762130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.762476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.763176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=3d7c1bdda0f040f4be95392dd75f4d16, user.id=d0ee218e69104872aa22674813a5ef39) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:56.764029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.765062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:56.771959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 97/194] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/role_assignments?user.id=d0ee218e69104872aa22674813a5ef39&scope.project.id=3d7c1bdda0f040f4be95392dd75f4d16&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:56.772824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.842135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.851610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cafeb3-223d-4b47-b4da-08ca4cafcc6a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.903952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.904601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.904879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.905149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.923179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a762787-9dd1-496d-8e5b-1ede77051187 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:56.923933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:56.930475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:56.931461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:42:56.932236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 98/195] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:56.932882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:56.943969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:56.986851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:56.986851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:56.986851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:56.986851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:56.987944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:56.988996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:56.990095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.001919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 98/196] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.002595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.101240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c39def-5142-4a09-b313-3a80333859c7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.139959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae2f9a-d3dc-414d-9f14-c7480371f4ef None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.141679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.143222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.143549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.143821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.184064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.184914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.185206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.185484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.186593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:57.186903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.188602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.189382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.202052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 99/197] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.202914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.256424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.257274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.259492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.330873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153f22f-c4f4-4d4d-81b0-577d40c54230 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.369154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.369789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.370066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.370389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.371485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.372930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.374779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 100/198] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.375383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.395670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 99/199] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:57.396515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.463339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.493867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbfab1-d199-4519-a098-6c13c15090e8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.495433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc66eb-b906-4371-a145-c4dac81ae59f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.532857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.533667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.534671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.535117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] PATH_INFO: `/v3/users/b369254e9f9141b58cefeaa8ca19c0b5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.536488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] RBAC: Authorizing `identity:get_user(user_id=b369254e9f9141b58cefeaa8ca19c0b5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.538169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.540101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 101/200] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/users/b369254e9f9141b58cefeaa8ca19c0b5 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.541093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.564957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.565613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.565886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.566151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.643016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.645128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.647954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77b865-659f-4a6a-89c2-50b73db69851 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.684340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.685349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.685828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.686317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.687720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.689371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.691583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 102/201] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.692641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.713450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.735691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:57.769503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 100/202] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:57.770710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.815267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7991-66db-4695-aa66-8c93451ac12b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.858764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.859840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.860225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.860602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.882414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbba6b-b7c6-4b49-addf-7a8a057412b1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:57.920425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:57.921355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:57.921964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:57.922461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:57.923500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=559064f4990b4266bb400290403e37a5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:57.924001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.925844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.936722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 101/203] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/role_assignments?user.id=559064f4990b4266bb400290403e37a5&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:57.938183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:57.961375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:57.963411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:57.997171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:58.055126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:58.130721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 103/204] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:42:58.139011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:58.152703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1a61-bbf3-416e-a99e-8ee57d07c858 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:58.256329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:58.256888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:58.257167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:58.257437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:58.263669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:58.265144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:58.266253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f96c-d8e6-41f3-93fe-048bb22e6330 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:58.267133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 102/205] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:57 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:58.275884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:58.305535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:58.306167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:58.306659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:58.306941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:58.308505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:58.309973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:58.312147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 104/206] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:42:58 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:58.319726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:58.334931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:58.469713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:58.492402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89268ea8-fe02-4c20-8756-85890ffdfdea None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:58.495859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce96d-3a1c-40b1-9be8-7567c8eac6e3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:58.528161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:58.528686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:58.528962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:58.529231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:58.529958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, scope.project.id=7713e09455654744919d00b7f24e4f3e, user.id=b369254e9f9141b58cefeaa8ca19c0b5) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:58.530260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:58.531722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb1730-674b-4494-9f85-4d0e23b58c6f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:58.539849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 105/207] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:58 2022] GET /identity/v3/role_assignments?user.id=b369254e9f9141b58cefeaa8ca19c0b5&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:58.561963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:58.562649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:58.562933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:58.563229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:58.564086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=559064f4990b4266bb400290403e37a5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:58.564398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:58.566222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb576a-3988-47ed-8fc2-953b6b9da72d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:58.573053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 103/208] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:42:58 2022] GET /identity/v3/role_assignments?user.id=559064f4990b4266bb400290403e37a5&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=da668845afcd4a95bf6de91627f699b4 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:59.036146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675d193-8800-4083-a5aa-2be43ba8966b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.036685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675d193-8800-4083-a5aa-2be43ba8966b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.036685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675d193-8800-4083-a5aa-2be43ba8966b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.038015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 106/209] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:42:59.043723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202f39-d2c2-4a21-b51c-1f76fa5ae576 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.044171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202f39-d2c2-4a21-b51c-1f76fa5ae576 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.044345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202f39-d2c2-4a21-b51c-1f76fa5ae576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.078951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b1202f39-d2c2-4a21-b51c-1f76fa5ae576 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:59.080253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b1202f39-d2c2-4a21-b51c-1f76fa5ae576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:59.157568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 104/210] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:42:59.184539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675d193-8800-4083-a5aa-2be43ba8966b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.225667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f8f76-c47f-4587-8bbf-16cb71ccc455 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.225947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f8f76-c47f-4587-8bbf-16cb71ccc455 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.226279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f8f76-c47f-4587-8bbf-16cb71ccc455 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.227655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 105/211] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:42:59.234145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.234427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.234757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.258586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:42:59.260202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:42:59.278623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:59.318963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675d193-8800-4083-a5aa-2be43ba8966b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.349721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 106/212] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:42:59.365923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.366599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.366874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.367211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.375754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.378306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:42:59.385299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:42:59.386397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:42:59.387247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 107/213] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:42:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:42:59.392102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.399535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:59.490739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2574c0-9ad2-4ccd-83f3-f9f1396c3832 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.492420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829a2e-ed30-41e7-86e3-9ff91810949e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.520308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:59.526495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.527200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.527582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.527946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.528703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:42:59.529119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:59.529476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.530271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.530686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.530831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:59.531261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.532679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:59.534925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:59.536656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 107/214] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:42:59.544243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.545008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 108/215] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:42:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:59.545925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.647166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ad8b1-af66-4cf6-9d83-07e2061ce7a1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.651336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:59.655526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0295-ca5f-4b12-81e1-f4df4b7f09f7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.685776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.687061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.687493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.687976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] PATH_INFO: `/v3/users/8b91e46f6f4a4561b15c28c6804a1955` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.697708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.698774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.699276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.699748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.700623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:59.702521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:59.704433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] RBAC: Authorizing `identity:get_user(user_id=8b91e46f6f4a4561b15c28c6804a1955)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:59.705948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:42:59.707817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 109/216] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:42:59 2022] GET /identity/v3/users/8b91e46f6f4a4561b15c28c6804a1955 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:42:59.715477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:42:59.729636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 108/217] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:42:59 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:42:59.778691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:42:59.835667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85bad-6fb1-49d9-b174-ae1a86f70335 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:42:59.875321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:42:59.875939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:42:59.876214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:42:59.876486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:42:59.949894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:42:59.951317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e54419-787a-4506-80a0-1b596d7a968c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:00.002156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:00.059023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 110/218] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:42:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:43:00.068142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:00.182595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683919-6e6e-43a9-ae41-c6312279eaec None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.183267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683919-6e6e-43a9-ae41-c6312279eaec None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.183746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683919-6e6e-43a9-ae41-c6312279eaec None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.185575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 111/219] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:00.193555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.194170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.194635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.194739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e88b70-bfa9-498b-972c-7f7f2150fd9e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:00.216889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:00.218478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:00.244700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:00.245359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.245637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.245907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.251818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:00.253836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:00.255750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 109/220] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:00.262654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:00.276916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:00.301722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 112/221] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:00.392506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:00.443596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8a389-2330-4271-afde-5fb98dba2b0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:00.499553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41638cc6-1a78-476b-85cf-650c70eded1b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:00.503578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:00.527038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:00.529730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.530320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.530793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.531779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=8b91e46f6f4a4561b15c28c6804a1955) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:00.532293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:00.534097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:00.541088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 110/222] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/role_assignments?user.id=8b91e46f6f4a4561b15c28c6804a1955&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:00.547112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:00.547112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.547112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.547112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.558485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:00.563541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:00.564512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:00.565870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 113/223] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:00.571710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:00.679871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:00.718025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8068cfa-c2d7-4507-b8c0-50e44c54f363 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:00.769968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:00.770510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.770788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.771171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.773516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:00.774009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:00.776487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff6fec-6f12-4bbe-a24c-4b71eeda9e9e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:00.791749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:00.797788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:00.876846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2127-11b3-491f-95ff-9ee3b133aa12 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.877037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2127-11b3-491f-95ff-9ee3b133aa12 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.877251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b2127-11b3-491f-95ff-9ee3b133aa12 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.878966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 112/225] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:00.884299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.884588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.884834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.923047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:00.924327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:00.930950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae04e1c-8376-4fb1-8153-34c8ab2b2a6a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:00.973500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:00.975204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:00.975698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:00.976170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:00.977627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:00.979336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:00.981816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 114/226] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:00.988613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:01.010384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:01.051589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 113/227] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:01.101205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:01.198659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e8f5f-4f92-43e0-82b7-bcbe07bb28c2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:01.214607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:01.288433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:01.289075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:01.289350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:01.289619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:01.291703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:01.293151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:01.300703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 115/228] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:00 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:01.304123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:01.326006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d8187-5c90-459c-b5e7-1584a3925ea0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:01.420458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:01.427712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:01.428380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:01.428664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:01.428937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:01.433479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f7a3d-cdbc-46a4-aa29-35cb042957e3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:01.454644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:01.469496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:01.470453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:01.471308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:01.472469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 114/229] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:01.475814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:01.476510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:01.476783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:01.477060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/559064f4990b4266bb400290403e37a5/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:01.481746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, user_id=559064f4990b4266bb400290403e37a5, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:01.483374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:01.484399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:01.501255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 116/230] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:01 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/559064f4990b4266bb400290403e37a5/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:01.659583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5216a-3950-4b45-82e3-9901bbdd8536 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:01.700982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:01.711495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:01.712198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:01.712507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:01.712740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:01.713424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:01.714144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:01.715156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:01.725150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 115/231] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:01.730933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:01.851892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19172ad6-7274-4361-abde-76101b6180ae None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:01.891710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:01.892669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:01.893265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:01.893732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:01.895659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:01.898296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:01.904589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 117/232] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:01 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:01.908092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:02.070933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252dbd-63f6-4a57-ac6c-79d7eaeaa734 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:02.116939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:02.117925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.117925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.118181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.119342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:02.120793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e1b47-9a6d-4d7f-b1ba-4d9ac8235663 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:02.122747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 116/233] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:01 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:02.128302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:02.147343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:02.266033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f7af-6061-460c-befa-397e096f6968 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:02.301854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:02.302550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.302828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.303125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/559064f4990b4266bb400290403e37a5/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.305414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, user_id=559064f4990b4266bb400290403e37a5, role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:02.306857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28807119-6590-4c98-b79c-f21d3da9668d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:02.326118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 118/234] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:02 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/559064f4990b4266bb400290403e37a5/roles/da668845afcd4a95bf6de91627f699b4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:02.392051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a5b3-19c6-4d55-8189-b11ef1bc6c77 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.392721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a5b3-19c6-4d55-8189-b11ef1bc6c77 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.393756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7a5b3-19c6-4d55-8189-b11ef1bc6c77 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.395425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 117/235] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:02.401351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013af7d-da38-49d0-a748-281df8b12fa6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.401657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013af7d-da38-49d0-a748-281df8b12fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.401906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013af7d-da38-49d0-a748-281df8b12fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.423216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b013af7d-da38-49d0-a748-281df8b12fa6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:02.424474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b013af7d-da38-49d0-a748-281df8b12fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:02.501114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:02.530714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 119/236] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:02.573516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a5b3-19c6-4d55-8189-b11ef1bc6c77 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:02.638404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:02.704633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7a5b3-19c6-4d55-8189-b11ef1bc6c77 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:02.728931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c8dc-9d57-4240-8190-dca0aa36f31f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.729233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c8dc-9d57-4240-8190-dca0aa36f31f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.729480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c8dc-9d57-4240-8190-dca0aa36f31f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.735215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 120/237] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:02.739133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.739417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.739692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.745962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:02.746593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.746868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.747159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.749244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:02.750690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:02.752286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 118/238] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:02.758590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:02.777268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:02.778568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:02.868603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963bb69-b93c-4fc0-a7ec-399e779770e1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:02.893688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 121/239] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:02.897764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:02.914130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:02.914742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:02.915040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:02.919582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:02.919582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:02.919582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea3dd8-21b7-4cf7-ae59-2d986dbc553c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:02.932775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:02.953804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 119/240] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:02 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:43:03.073330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed143f89-4fa3-4891-ae84-61f55c5ac80e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.111808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.131188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.131849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.132129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.132402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.135390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2fa46-563f-4a2b-a627-06a611fec500 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.136271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2fa46-563f-4a2b-a627-06a611fec500 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.136723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2fa46-563f-4a2b-a627-06a611fec500 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.138563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 120/241] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:03.146462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.147035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.147517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.159397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:03.164653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:03.165622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:03.166410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 122/242] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:03.173017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:03.174096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.174850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:03.270877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 121/243] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:03.293221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.317075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f84ccb-638a-49e8-bb1c-39a41b45ea4a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.373170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.374192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.374719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.375220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.376079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:03.376588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:03.380477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:03.392808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 123/244] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:03.398832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.428266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231652a-f93b-40bd-bb55-f16ec5afb264 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.441937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.474773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.489058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.489779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.490133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.490717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.516547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:03.526331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:03.527298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:03.528071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:03.530708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 122/245] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:03.537728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.556003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1e382-bbbf-40dc-a810-3174e665eba9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.594668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.595454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.595802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.596264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] PATH_INFO: `/v3/users/e12d28fdcd1441efa7328ffec688872b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.613194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] RBAC: Authorizing `identity:get_user(user_id=e12d28fdcd1441efa7328ffec688872b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:03.614875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:03.617090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 124/246] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/users/e12d28fdcd1441efa7328ffec688872b => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:03.622748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.656163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de8b99-d972-45dc-b47d-3ee66943b849 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.695092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.696062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.696558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.697034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.697896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:03.698390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:03.700022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:03.709208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 123/247] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:03.715041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.738534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3663f5c-8d5f-4379-92bc-e77f0d1bdd9d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.741076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.746709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.782474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.783144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.783426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.783735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.826082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb455b-15f4-4705-bbfa-759531d3bda1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:03.860256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:03.861743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:03.865066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:03.865968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:03.866466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:03.866941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] PATH_INFO: `/v3/users/8b91e46f6f4a4561b15c28c6804a1955` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:03.868447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] RBAC: Authorizing `identity:get_user(user_id=8b91e46f6f4a4561b15c28c6804a1955)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:03.870185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:03.872193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 124/248] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/users/8b91e46f6f4a4561b15c28c6804a1955 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:03.879171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.975319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 125/249] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:03.978914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:03.983879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:03.990029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b17a7a-1fd3-407f-9f0e-fd2c98e1c219 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:04.015389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:04.074775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:04.074775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.074775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.074775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.079905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:04.081500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:04.085983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 125/250] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:04.092647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:04.182740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9701-c29d-48d1-8410-7e56bed0abb8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:04.222529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:04.223819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.224346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.224819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.228281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:04.229975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:04.232162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 126/251] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:03 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:04.239201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:04.344699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc4bac-7206-4bfb-8335-6b2aa13d8b39 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:04.405425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:04.406092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.406535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.406818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/8b91e46f6f4a4561b15c28c6804a1955/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.409119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, user_id=8b91e46f6f4a4561b15c28c6804a1955, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:04.410651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394924d-d338-4657-a77b-c0e231be156b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:04.439004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:04.442442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 126/252] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:04 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/8b91e46f6f4a4561b15c28c6804a1955/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:04.486214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac25501-28ec-45ea-900c-7f82a5c51009 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:04.529615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c2bf-ff9f-4115-99ab-91853ce9c36a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.529899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c2bf-ff9f-4115-99ab-91853ce9c36a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.530175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c2bf-ff9f-4115-99ab-91853ce9c36a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.531809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 127/253] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:04.550644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.550935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.551227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.554368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:04.555199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.555663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.556096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.557027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=e12d28fdcd1441efa7328ffec688872b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:04.557544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:04.561440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:04.571410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 127/254] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:04 2022] GET /identity/v3/role_assignments?user.id=e12d28fdcd1441efa7328ffec688872b&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:04.583467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:04.584697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:04.616617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:04.682099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 128/255] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:04.705812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:04.818679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a6698-47ae-4384-8eab-f12002cb8e9f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:04.855420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:04.856663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:04.857134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:04.857590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:04.860901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:04.870722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:04.876377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:04.877270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:04.878122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:04.880502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 128/256] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:04.887563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:04.998465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898d543-485f-4403-84bd-c3cfd0085be9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:05.044978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:05.045846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.046350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.046812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.047644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:05.048085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:05.049636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:05.062918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 129/257] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:05.071002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:05.193516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06047c7a-5ede-4e90-ac58-757d3425ace6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:05.228487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:05.229118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.229447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.229720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.230896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:05.232747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:05.234317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 129/258] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:05 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:05.240425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:05.259611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:05.348585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab080c-a677-4b00-8ade-c4cf9bdae498 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:05.389202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:05.389813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.390097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.390366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.468563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:05.470089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9c1e8-3bb8-4c81-8bc6-ca530e0c108c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:05.492616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:05.584498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 130/259] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:05.589932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:05.658303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:05.699226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673cae4-cfa6-4c49-9fc1-05e1e678e31b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:05.715613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96456a6f-7289-4ffe-9b43-37eba0652304 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.715905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96456a6f-7289-4ffe-9b43-37eba0652304 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.716148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96456a6f-7289-4ffe-9b43-37eba0652304 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.717485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 131/260] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:05.723239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.723557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.723865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.739739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:05.740378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:05.740658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:05.740928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:05.742205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:05.743748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:05.745400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 130/261] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:05 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:05.751182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:05.769846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:05.771103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:05.894721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 132/262] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:05.923314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb015c-3a0e-4d04-8d8e-5b11876b2c4c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:05.930878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:05.990299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:06.008440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:06.009269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.009759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.010236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.011619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=559064f4990b4266bb400290403e37a5) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:06.011935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:06.014492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0a13e-01c6-461b-8c6d-9ec5f61c1af7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:06.037336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:06.039176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 131/263] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:05 2022] GET /identity/v3/role_assignments?user.id=559064f4990b4266bb400290403e37a5&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:06.108311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ef058-852c-4948-966c-15c7ea4afa83 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:06.160750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b7b6-edda-44cc-9eef-a543bf30fcde None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.161167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b7b6-edda-44cc-9eef-a543bf30fcde None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.161338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b7b6-edda-44cc-9eef-a543bf30fcde None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.161818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:06.162468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.162739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 132/264] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:06.162856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.163132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.184259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:06.185273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.185562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.185840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:06.189022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:06.190440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:06.191271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 133/265] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:06.199186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:06.226891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:06.228147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:06.387647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 133/266] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:06.416039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125afa9-4c91-4321-96b4-ec00c16f03f4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:06.427333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:06.432475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:06.494382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:06.494914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.495228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.495506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.496361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:06.496771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:06.498167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:06.516977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 134/267] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:06.523327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:06.685639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bae86c-0952-4977-acf5-64c1b9b34e49 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:06.693995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c4e9-3751-4596-bd78-0a8dfd5f8bfc None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:06.730190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:06.731488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.731973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.732484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] PATH_INFO: `/v3/users/559064f4990b4266bb400290403e37a5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.733955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] RBAC: Authorizing `identity:get_user(user_id=559064f4990b4266bb400290403e37a5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:06.735685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f53a84-4d04-452b-8ddf-f1415fc0acd8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:06.738017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 135/268] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:06 2022] GET /identity/v3/users/559064f4990b4266bb400290403e37a5 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:06.739346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0051-df54-4bb3-9557-dcb127a94032 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.739834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0051-df54-4bb3-9557-dcb127a94032 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.740327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0051-df54-4bb3-9557-dcb127a94032 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.741808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 136/269] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:06.744291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d0051-df54-4bb3-9557-dcb127a94032 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:06.747357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:06.759843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:06.760829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.761311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.761797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.763229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:06.764896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffae66-e923-4ba7-ac2f-479b41d26a1a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:06.767025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 134/270] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:06.768394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:06.768894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:06.769368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:06.790061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:06.791769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:06.851339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:06.873522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 135/271] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:06.874650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:06.993717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6e39b-6cc3-4c67-850a-bdba810661be None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.000085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d0051-df54-4bb3-9557-dcb127a94032 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.021321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.034581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.035223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.035502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.035773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.036589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.038174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.052969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.054103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.054694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.055280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.072979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 136/272] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:06 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:07.074075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.141749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.144159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.189440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.258307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cfc47-87a6-43e7-af46-c62db5722219 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.292143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 137/273] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:43:07.299091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.305365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.306063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.306516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.306798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.321733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:07.328509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:07.329426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:07.330214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 137/274] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:07.337859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.392645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.429111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22eca2d-7a9b-482d-b882-d4f466b53c00 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.468879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d751a-9f79-435e-b1e6-1f07bd21ddf1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.478925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.479650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.480037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.480317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.481839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.483923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.488939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 138/275] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:07.495058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.510330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.511257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.511742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.512026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.512478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.513266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:07.513572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.516154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.528194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 138/276] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:07.534152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.641346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a727-2f74-4097-80b9-3972c1e78971 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.660266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9f5a-231f-48f4-b734-f82ce4751c43 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.661833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.681607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.682320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.682663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.682940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.683783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=559064f4990b4266bb400290403e37a5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:07.684093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.685643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2036d0-df88-4c5c-86fa-2114881b8340 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.692337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 139/277] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/role_assignments?user.id=559064f4990b4266bb400290403e37a5&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=da668845afcd4a95bf6de91627f699b4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:07.693703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411112-b228-4988-8c46-42ed8c73dee7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.693995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411112-b228-4988-8c46-42ed8c73dee7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.694571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411112-b228-4988-8c46-42ed8c73dee7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.696235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 140/278] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:07.701607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7074e4-75dd-425e-88a5-caa3e1fdca75 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.702691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7074e4-75dd-425e-88a5-caa3e1fdca75 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.702962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7074e4-75dd-425e-88a5-caa3e1fdca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.711094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.711790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.712041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.712320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] PATH_INFO: `/v3/users/deaa67fa0ff244648b0b4eb557b1533e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.728257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3f7074e4-75dd-425e-88a5-caa3e1fdca75 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:07.730016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f7074e4-75dd-425e-88a5-caa3e1fdca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:07.730511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] RBAC: Authorizing `identity:get_user(user_id=deaa67fa0ff244648b0b4eb557b1533e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.732312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d49974-a504-42d3-b03c-ea0d06b3dec2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.735265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 139/279] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/users/deaa67fa0ff244648b0b4eb557b1533e => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:07.737877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179b06-8059-4d77-86b8-bf3742680b86 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.741807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179b06-8059-4d77-86b8-bf3742680b86 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.742464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8179b06-8059-4d77-86b8-bf3742680b86 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.744206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 140/280] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:07.746954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179b06-8059-4d77-86b8-bf3742680b86 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.832399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:07.847100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 141/281] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:07.848708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.849959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.850444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.878937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:07.880818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:07.920065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8179b06-8059-4d77-86b8-bf3742680b86 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:07.964274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:07.965222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:07.965725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:07.966246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:07.967254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, user.id=deaa67fa0ff244648b0b4eb557b1533e, scope.system=all) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:07.967766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:07.969483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:07.977414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 142/282] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:07.978331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:07.978814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 141/283] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/role_assignments?user.id=deaa67fa0ff244648b0b4eb557b1533e&scope.system=all&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:08.002664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.037268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.119510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdfb0c-1c90-43f8-9f52-7a8776bb65b1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.140644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10197a97-1d85-4acb-b5fe-1499b6a409a4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.156626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.168510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.169166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.169448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.169723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.186254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:08.187279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.188134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.188633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.189132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:08.189304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:08.190731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:08.191745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 143/284] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:08.197823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.212118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:08.216938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:08.217846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:08.218631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:08.223155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 142/285] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:08.226089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.325814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feac8a-f250-4b7d-8df2-7c02b10a94a1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.378356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.378896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.380849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.381138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.381836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:08.386949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:08.388500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:08.406078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 144/286] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:08.411353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.412332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.445485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd81c1a-3baa-41f9-a502-7a0a5fe0ea1b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.494296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.537017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.537626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.537899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.538164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.538827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:08.539153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:08.540533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:08.549708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69353-7beb-48ef-9f3c-13466a502e8b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.562259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 143/287] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:08.571952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.595299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.595579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.595850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] PATH_INFO: `/v3/users/8b91e46f6f4a4561b15c28c6804a1955` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.597696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] RBAC: Authorizing `identity:get_user(user_id=8b91e46f6f4a4561b15c28c6804a1955)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:08.599591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:08.601423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 145/288] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/users/8b91e46f6f4a4561b15c28c6804a1955 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:08.609616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.692708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a87c7-0569-4255-b880-e76bd296661c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.741058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.741838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.742066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.742249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] PATH_INFO: `/v3/users/e12d28fdcd1441efa7328ffec688872b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.743376 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=e12d28fdcd1441efa7328ffec688872b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:08.744826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:08.746805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 144/289] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/users/e12d28fdcd1441efa7328ffec688872b => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:08.751617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:08.791339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.833334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db75a2-93d8-4666-bdff-1cb2528a58e3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:08.866033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:08.900734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:08.901303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:08.901581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:08.901843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:08.947608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c4f60-988b-4441-9e4f-521909761ec0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:09.015684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:09.016312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.016586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.016855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.018589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:09.021890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:09.023540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 145/290] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:09.031301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:09.038459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:09.041391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:09.131346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:09.144253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a8175-1e86-4901-b746-97d2c63052cf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:09.178864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:09.179565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.179844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.180120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/e12d28fdcd1441efa7328ffec688872b/roles/5e4ea42893434cc9b1df29922b3cab0d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.182085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, user_id=e12d28fdcd1441efa7328ffec688872b, role_id=5e4ea42893434cc9b1df29922b3cab0d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:09.183679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:09.202535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 146/291] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:09 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/users/e12d28fdcd1441efa7328ffec688872b/roles/5e4ea42893434cc9b1df29922b3cab0d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:09.257836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 146/292] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:09.263626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:09.376940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6287cdf-f7f3-46d9-add8-7bb41852cf75 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:09.415508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:09.416155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.416427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.416698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.417913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:09.419443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e916c9-facd-4443-b1e8-83c9c4d60192 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:09.421259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 147/293] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:09 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:09.427271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:09.447485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:09.538190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09e96-75b4-4e42-b07c-434007935e44 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:09.584317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:09.584939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.585219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.585497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.586235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=8b91e46f6f4a4561b15c28c6804a1955) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:09.586539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:09.588115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c549a4-5ef6-4129-b182-f1eaec281e3c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:09.601640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 147/294] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:09 2022] GET /identity/v3/role_assignments?user.id=8b91e46f6f4a4561b15c28c6804a1955&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:09.761060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8333b-0c0e-4741-b111-a70e678d040b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.761568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8333b-0c0e-4741-b111-a70e678d040b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.761678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8333b-0c0e-4741-b111-a70e678d040b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.763070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 148/295] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:09.768628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:09.768906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:09.769179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:09.794495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:09.795624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:09.885923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 148/296] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:09.911505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8333b-0c0e-4741-b111-a70e678d040b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:09.928897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.039005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8333b-0c0e-4741-b111-a70e678d040b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.092229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.092869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.093148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.093423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.106314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f21475a5-531c-46e1-9853-7eee48c69460 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:10.112310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:10.113193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:10.113978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 149/297] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:10.119663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.143001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.208289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dea8-7d5f-4da6-ad92-94e959f8b633 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.208916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dea8-7d5f-4da6-ad92-94e959f8b633 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.209393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dea8-7d5f-4da6-ad92-94e959f8b633 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.211322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 150/298] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:10.219745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.220344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.220829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.241650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:10.243260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:10.249590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1039-4a61-4294-8d2d-3003cb12626f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.290247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.291684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.291968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.292247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.292917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:10.293230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:10.294637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:10.304693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 149/299] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:10.310289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.327772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.339450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 151/300] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:10.364385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.415212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.449811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46ef60-c2db-4093-97a9-8ff12a9082c1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.473782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f0b86-e030-4579-a0a1-8c94298ed9f6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.496022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.496797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.497181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.498132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] PATH_INFO: `/v3/users/cd45179a35c9427ab03a6a5cebe9da80` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.514274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.515165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.515662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.516076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.516421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] RBAC: Authorizing `identity:get_user(user_id=cd45179a35c9427ab03a6a5cebe9da80)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:10.517467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:10.518226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:10.519284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:10.520139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 150/301] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:10 2022] GET /identity/v3/users/cd45179a35c9427ab03a6a5cebe9da80 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:10.526098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.527095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 152/302] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:10.530225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.614476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.640652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6d713-7420-4780-94df-4fe7ac9a03ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.651886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb020dff-56ba-4105-b5cb-6c739eb4d282 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.676496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.681166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.682209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.682668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.682940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.684041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, user.id=cd45179a35c9427ab03a6a5cebe9da80, scope.system=all) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:10.684355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:10.685793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b55523-cb30-4d7f-be91-da6481b5e0cc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:10.690800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.698679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 151/303] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 9 17:43:10 2022] GET /identity/v3/role_assignments?user.id=cd45179a35c9427ab03a6a5cebe9da80&scope.system=all&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:10.698867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.698867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.698867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] PATH_INFO: `/v3/groups` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.699104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73cb6-0d19-4ed9-82d1-fc1b4176a23f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.699172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:10.699238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73cb6-0d19-4ed9-82d1-fc1b4176a23f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.699238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e73cb6-0d19-4ed9-82d1-fc1b4176a23f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.699364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:10.699430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 152/304] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:10.704292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.704627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.704851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:10.723462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 153/305] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Aug 9 17:43:10 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:10.726417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:10.727723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:10.811053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 153/306] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:10.833764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:10.954114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf9d43-62f2-44da-aa19-3e300a9586bf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:10.982911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:10.992028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:10.992674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:10.993291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:10.993802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.006009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:11.011289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:11.012205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:11.012964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 154/307] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:11.019138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:11.136744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62511737-47ce-48d3-b7bb-6437ac5c63e8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:11.166222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:11.177145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:11.178075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.178692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.179198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.180064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:11.180558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:11.182343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:11.193660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 154/308] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:11.203455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:11.315963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b04f1-eb47-455f-8f69-4bfecacf1fee None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:11.360870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:11.361507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.361813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.362087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] PATH_INFO: `/v3/users/deaa67fa0ff244648b0b4eb557b1533e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.363263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] RBAC: Authorizing `identity:get_user(user_id=deaa67fa0ff244648b0b4eb557b1533e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:11.364712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51afd-39e3-4a18-bdcb-9986e1745c83 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:11.366575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 155/309] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:11 2022] GET /identity/v3/users/deaa67fa0ff244648b0b4eb557b1533e => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:11.372265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:11.485469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732018cd-e9c1-4e34-928e-7fbd48d6bd3d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:11.524270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3832d-f75e-4445-88ea-693e0f8af69a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.524615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3832d-f75e-4445-88ea-693e0f8af69a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.524711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:11.524810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3832d-f75e-4445-88ea-693e0f8af69a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.525274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.525551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.525821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] PATH_INFO: `/v3/system/users/deaa67fa0ff244648b0b4eb557b1533e/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.526401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 156/310] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:11.527413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=deaa67fa0ff244648b0b4eb557b1533e, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:11.528903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:11.532110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.532383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.532654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.539727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 155/311] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Aug 9 17:43:11 2022] PUT /identity/v3/system/users/deaa67fa0ff244648b0b4eb557b1533e/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:11.564064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:11.566045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:11.633975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:11.654138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 157/312] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:11.677186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:11.737208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:11.793076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822905f0-98ce-48eb-9acd-23324fbfe9d1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:11.830404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:11.831468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:11.831965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:11.832513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:11.843844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:11.849500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:11.850360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:11.851165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:11.853718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 156/313] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:11.859676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:11.968810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309dc8b-83cd-47e6-9f17-3910e5f8633a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:12.006595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:12.007311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.007592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.007862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.008547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:12.008849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:12.010266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:12.019555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 158/314] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:12.025501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:12.135954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02682528-1ea9-41f1-8253-ff91317a80a9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:12.149488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:12.171585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:12.172507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.173000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.173484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] PATH_INFO: `/v3/users/e12d28fdcd1441efa7328ffec688872b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.175403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] RBAC: Authorizing `identity:get_user(user_id=e12d28fdcd1441efa7328ffec688872b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:12.177037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:12.178795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 157/315] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:12 2022] GET /identity/v3/users/e12d28fdcd1441efa7328ffec688872b => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:12.188265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:12.241583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:12.293825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74819a95-33fe-4978-b499-3fb84e07b031 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:12.329048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:12.329657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.329939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.330238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.410787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:12.412281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:12.461379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:12.519337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 159/316] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:12.521481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:12.623846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b2153-47ff-48a7-92a2-6bcb5560258b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:12.664061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:12.665025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.665510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.665978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.667317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:12.668961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff39d2e-ecaa-4a43-9b6d-79150dbe5ef4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:12.670780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 158/317] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:12 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:12.676877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:12.783196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84391-895a-43f0-a5c4-57ff1a5fa010 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:12.809267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad679ac-b817-4106-b187-09decafb9dfb None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.809777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad679ac-b817-4106-b187-09decafb9dfb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.809850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad679ac-b817-4106-b187-09decafb9dfb None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.812179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 159/318] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:12.817522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4e871-b226-4a78-9861-f27ff551ed0c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.817800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4e871-b226-4a78-9861-f27ff551ed0c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.818068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4e871-b226-4a78-9861-f27ff551ed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.820958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:12.821568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:12.821842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:12.822120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:12.822852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc, user.id=e12d28fdcd1441efa7328ffec688872b) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:12.823232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:12.824645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:12.831774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 160/319] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:12 2022] GET /identity/v3/role_assignments?user.id=e12d28fdcd1441efa7328ffec688872b&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:12.838669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-69b4e871-b226-4a78-9861-f27ff551ed0c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:12.840107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-69b4e871-b226-4a78-9861-f27ff551ed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:12.893938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:12.921210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 160/320] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:12.943523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:12.961979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.013053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c649a-d2b5-45f4-b502-7a357a4ae1a6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.013368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c649a-d2b5-45f4-b502-7a357a4ae1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.013620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c649a-d2b5-45f4-b502-7a357a4ae1a6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.016049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 161/321] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:13.021183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.021468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.046954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:13.048272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:13.054756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a65b79-7ef4-46be-aa71-ca3fd430877d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.104097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.105228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.105710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.106461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.121984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:13.127732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:13.128631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:13.129412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 161/322] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:13.135310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.147406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 162/323] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:13.169110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.216554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.250400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2921fe-ff5c-4cc8-8dd6-50d35d682e6e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.280958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe4905-c042-4b92-a211-54f58c7587c1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.287461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.288067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.288335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.289034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.289700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:13.289999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:13.290517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.291832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:13.301901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 162/324] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:13.306925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.321553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.322188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.322562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.322831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.323919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:13.325353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:13.327234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 163/325] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:13.332277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.436752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a088d3-581c-4e2c-9f3e-88bf0d334c1e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.446935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac3199-ab0e-4eb5-baa6-3b1e08a65697 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.473171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.473719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.473994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.474262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] PATH_INFO: `/v3/users/cd45179a35c9427ab03a6a5cebe9da80` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.475468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] RBAC: Authorizing `identity:get_user(user_id=cd45179a35c9427ab03a6a5cebe9da80)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:13.476904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:13.478463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 163/326] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:13 2022] GET /identity/v3/users/cd45179a35c9427ab03a6a5cebe9da80 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:13.482810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.483740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.484165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.484406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.484844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] PATH_INFO: `/v3/groups` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.485630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:13.487349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.488139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86543222-94ea-420f-aee0-a95a72a010ba None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:13.519009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 164/327] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Aug 9 17:43:13 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:43:13.597221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d46f50-1c81-42a6-93c7-e99e4c9327a6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:13.634595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:13.635284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.635561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.635831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] PATH_INFO: `/v3/system/users/cd45179a35c9427ab03a6a5cebe9da80/roles/5e4ea42893434cc9b1df29922b3cab0d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.637429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cd45179a35c9427ab03a6a5cebe9da80, role_id=5e4ea42893434cc9b1df29922b3cab0d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:13.638826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37955779-5e96-4537-9bb1-9e3806538f64 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:13.655676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 164/328] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Aug 9 17:43:13 2022] PUT /identity/v3/system/users/cd45179a35c9427ab03a6a5cebe9da80/roles/5e4ea42893434cc9b1df29922b3cab0d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:43:13.713579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18940b0-81b8-4de8-b361-486a713cd37d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.713910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18940b0-81b8-4de8-b361-486a713cd37d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.714136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18940b0-81b8-4de8-b361-486a713cd37d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.715552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 165/329] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:13.721954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:13.722233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:13.722510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:13.743185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:13.745130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:13.818313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.828352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 165/330] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:13.851802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18940b0-81b8-4de8-b361-486a713cd37d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:13.891809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:13.966728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18940b0-81b8-4de8-b361-486a713cd37d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:14.009538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:14.010096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:14.010372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:14.010639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:14.023569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:14.028455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:14.029346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:14.030127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:14.035068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 166/331] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:14.036673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:14.174911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f50e7-aaac-4422-a4a1-14099c379e36 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:14.221350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:14.222328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:14.222819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:14.223308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:14.224160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:14.224648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:14.226292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:14.235949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 166/332] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:14.242072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:14.391355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:14.397792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7836-092f-46c3-8ba5-9bf3500f23de None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:14.447909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:14.448580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:14.448859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:14.449133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] PATH_INFO: `/v3/users/deaa67fa0ff244648b0b4eb557b1533e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:14.456292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] RBAC: Authorizing `identity:get_user(user_id=deaa67fa0ff244648b0b4eb557b1533e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:14.457799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fa74f-3acd-4ee2-a120-808035a61568 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:14.460151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 167/333] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:14 2022] GET /identity/v3/users/deaa67fa0ff244648b0b4eb557b1533e => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:14.467213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:14.557484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:14.607719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be3444-84b7-4832-95a0-f1df01b27aed None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:14.658124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:14.658690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:14.658986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:14.659265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:14.660000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] RBAC: Adding query filter params (role.id=b424450c766b4e32b0c1dbb8d5088e4d, user.id=deaa67fa0ff244648b0b4eb557b1533e, scope.system=all) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:14.660305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:14.661750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e58b9-d549-4d55-8984-cac55b5892cd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:14.677775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 167/334] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 9 17:43:14 2022] GET /identity/v3/role_assignments?user.id=deaa67fa0ff244648b0b4eb557b1533e&scope.system=all&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:16.144328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d76c5-3ba7-492f-853c-094611f01e5e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.144862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d76c5-3ba7-492f-853c-094611f01e5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.144862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d76c5-3ba7-492f-853c-094611f01e5e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.146450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 168/335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:16.165057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c275-7e0f-4563-bfd0-f1ab2a277dda None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.165562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c275-7e0f-4563-bfd0-f1ab2a277dda None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.165634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028c275-7e0f-4563-bfd0-f1ab2a277dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.187961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8028c275-7e0f-4563-bfd0-f1ab2a277dda None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:16.189156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8028c275-7e0f-4563-bfd0-f1ab2a277dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:16.278690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 168/336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:16.292497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7760de-86bd-4830-a6a3-384e57f5fb67 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.292774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7760de-86bd-4830-a6a3-384e57f5fb67 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.293243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7760de-86bd-4830-a6a3-384e57f5fb67 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.294734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 169/337] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:16.300675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.300962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.301244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.324078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:16.325218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:16.354519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7760de-86bd-4830-a6a3-384e57f5fb67 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:16.390373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:16.445439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 169/338] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:16.506820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:16.581354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7760de-86bd-4830-a6a3-384e57f5fb67 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:16.656535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:16.657172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.657447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.657715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.684619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:16.699060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceae371-3e72-4a8b-b1c6-8cc836dbea7a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:16.702612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:16.703535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:16.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:16.704915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 170/339] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:16.722367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:16.790087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:16.799858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:16.800517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.800797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.801045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.828470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a7af082c-162f-4415-b942-99ace19efe5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:16.838809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:16.841181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:16.843134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20f7ac-12fb-4537-bcbc-4fbc169fd3c1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.844880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20f7ac-12fb-4537-bcbc-4fbc169fd3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.844880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20f7ac-12fb-4537-bcbc-4fbc169fd3c1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.851079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 171/341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:16.851915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20f7ac-12fb-4537-bcbc-4fbc169fd3c1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:16.914450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa9385-235e-45cc-b549-bfaba110e7c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:16.953844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:16.954672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.954929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.955854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.956540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:16.956843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:16.958298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9167c5f-3b16-4d2e-8c4a-6ccaee2003f9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:16.961507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:16.970767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 171/342] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:16.971842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:16.972125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:16.972398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:16.997645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:16.998841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:17.079349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 172/343] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:17.080461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.100896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.136627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20f7ac-12fb-4537-bcbc-4fbc169fd3c1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.194729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a2230-72e4-4d5d-a5ac-f159df01a3c2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.229382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.230210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.230486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.230755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.231585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.231703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:17.232000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.232162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.232437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.232713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.233488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.250435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 172/344] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:17.251307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.310704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.312288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.332147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.391768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.397411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a201e-17ee-4ef8-b956-b7610e3434ba None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.442411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.443346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.443629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.443944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.457213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ad73048a-07f5-4433-81dd-8079bdd3ec3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:17.462685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:17.464821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:17.466879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:17.468369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 173/345] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:17.468519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 173/346] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:17.468657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.468809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e60f-5b63-49e3-834d-1ffd952e9728 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.468809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e60f-5b63-49e3-834d-1ffd952e9728 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.468809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42e60f-5b63-49e3-834d-1ffd952e9728 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.473402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 174/347] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:17.474331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42e60f-5b63-49e3-834d-1ffd952e9728 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.576320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c1a70-5af0-4df3-8992-b47fb058b403 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.586471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42e60f-5b63-49e3-834d-1ffd952e9728 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.593890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.625045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.625677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.625954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.626220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] PATH_INFO: `/v3/users/cd45179a35c9427ab03a6a5cebe9da80` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.627904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cd45179a35c9427ab03a6a5cebe9da80)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.629341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.631324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 174/348] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/users/cd45179a35c9427ab03a6a5cebe9da80 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:17.632022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.632795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.633423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.633696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.634019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.635490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.638355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0972066-58d5-4cee-97f9-ffd2c5e3c403 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.640389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 175/349] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:17.641555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.641885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.642097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.664979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:17.666415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:17.691167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.744116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 176/350] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:17.745022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.768888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42c615-d908-4590-9453-5a913217f5c6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.821325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.822034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.822360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.822679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.823437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:17.823735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.825115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.837362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 175/351] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:17.838203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.881927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd5cc-2791-400f-bfaa-7728e84eeb76 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.893606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:17.936210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:17.936833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.937113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.937382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.938110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] RBAC: Adding query filter params (role.id=5e4ea42893434cc9b1df29922b3cab0d, user.id=cd45179a35c9427ab03a6a5cebe9da80, scope.system=all) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:17.938414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:17.939974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48a35e-5e6b-4d88-9d57-e2a50c02c422 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:17.957838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 177/352] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/role_assignments?user.id=cd45179a35c9427ab03a6a5cebe9da80&scope.system=all&role.id=5e4ea42893434cc9b1df29922b3cab0d => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:17.957838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc1924-ccf5-4dea-9f83-54df179c564d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:17.957838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc1924-ccf5-4dea-9f83-54df179c564d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:17.957838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc1924-ccf5-4dea-9f83-54df179c564d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:17.957838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 178/353] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:17.958469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc1924-ccf5-4dea-9f83-54df179c564d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:17.977230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78c8e-8172-4bd0-870f-81e84b958fc8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:17.981286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.018608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.019276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.019657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.019945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.030481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.032243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.034735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 176/354] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:18.035643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.105266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc1924-ccf5-4dea-9f83-54df179c564d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.160755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.161765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.162106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.162384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.177885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a91be20e-f436-4bcf-931e-eae668fc8445 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:18.184917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:18.185816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:18.186752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 179/355] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:18.188129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.188405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.188672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.189972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15600027-bec4-4b4b-a5d7-246dd6c106f5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.203514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.212946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:18.214142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:18.233023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.233639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.233994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.234241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.312428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 180/356] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:18.313283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.316967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.321683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.323411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.435931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d5057-adad-48e9-ba3e-563d94d54224 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.443787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 177/357] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:18.445062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.460725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.477992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.478604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.478879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.479175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.479896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=7713e09455654744919d00b7f24e4f3e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:18.480193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.482274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.488697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 181/358] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:18.488697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.591431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.592242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953aa2f-4867-4e5b-87d7-21f5b3364b9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.638099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944b60-bcf1-4d5e-8561-58a8fb1efcfe None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.649603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.650239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.651181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.651455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.652969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:18.652969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.654920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0bf59-f06a-4231-8288-46b3782f0279 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 178/359] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df476-1950-4f82-8d94-121a58bf0323 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df476-1950-4f82-8d94-121a58bf0323 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7df476-1950-4f82-8d94-121a58bf0323 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 179/360] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:18.670105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7df476-1950-4f82-8d94-121a58bf0323 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.696998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.697545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.699692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.699692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.713506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:18.719102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:18.720224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:18.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 182/361] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:18.722099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:18.792653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.841788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7df476-1950-4f82-8d94-121a58bf0323 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.855192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50312b0-7e08-40bc-9310-dbb3df9f8b76 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:18.881189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.882275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.882275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.882456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.883646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.885100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dc179-03d6-4c86-921e-76789e99895c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.886057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:18.887374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 180/362] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:18.888304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.888418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.888853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.897702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:18.898317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:18.898597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:18.898873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:18.909760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:18.911103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:18.981528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:18.983039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:18.996933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 181/363] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:18.997634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.091521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.131588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3aff13-163a-43db-9803-82c77b4be252 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.140777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 183/364] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:19.142504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.179665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.186604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.189040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.189040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.189040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.189040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:19.189040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.190144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.200157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 182/365] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:19.205315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.269248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba561-35fe-432c-9ab1-3c534d340718 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.312475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.313127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.313373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.313641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.314922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.316549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.318223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 184/366] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:19.324287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.349176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c4e95-164e-4294-bccc-1e4be0c412be None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.385187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.394844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.395537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.395840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.396109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.409282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:19.414316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:19.415223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:19.416028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 183/367] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:19.417185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.448145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.453065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b8df2-ac0b-4f03-a976-e0a0586ff765 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.496692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.497319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.497593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.497861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.499569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.500989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.502903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 185/368] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:19.509886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.540993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3f1bd-3f1f-45d2-8ef2-747cd123abc5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.578514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.579893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.580962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.581452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.647371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35c546-430d-464f-85a6-7cafadbab2c1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.664958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.677312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.678788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.699018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.699642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.699921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.700856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.700978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=7713e09455654744919d00b7f24e4f3e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:19.701201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.702836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.710545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 186/369] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=da668845afcd4a95bf6de91627f699b4 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:19.711652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.753356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:19.832612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 184/370] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:19.833730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.855546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169408c-cadc-4f34-b949-77e1732b5a74 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.897930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:19.898872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:19.899364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:19.899807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:19.900660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:19.901130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:19.902800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:19.912763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 187/371] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:19.913952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:19.954600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448de5f2-5309-4541-ac57-0dc28324906d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:19.978562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.001425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.002364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.002718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.004908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.006420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.008117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.010213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 185/372] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.011473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.025982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.039712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068cc38-512a-4368-9d87-949464f420c8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.080705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.081623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.081902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.082168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.083372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.085244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.087231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 188/373] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:19 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.088158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.145314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc72a8-8dbb-4869-8ebd-6449952533ed None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.196463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.197210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.197534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.197816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.232167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2c51-0ab4-4414-a6e1-7443e63cb17e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.255359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.288174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.288797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.289078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.289354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.290384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:20.290696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.292727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.302934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 189/374] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.302934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.310764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.312596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.351378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.443148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af43d0-02c9-4b3c-b582-a24617a05053 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.444332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 186/375] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:20.456271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.504226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.505915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.506580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.508282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.510531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.512620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.515338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 190/376] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.525186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.601373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.655723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc5c9f-0e70-4913-bce5-8687cedc01d7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.690595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.702225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998e76f-b99a-4bc1-adef-d4baf57950c6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.724264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.724907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.725182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.725451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] PATH_INFO: `/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.731900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] RBAC: Authorizing `identity:get_project(project_id=5aa4b373e99e4d74b2148fd834c1abbb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.733363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.735393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 187/377] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb => generated 344 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.741310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.750316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.750926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.751283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.751560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.752349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:20.752656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.754399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95685a-f88f-4325-91e4-3837604727ac None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.760923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 191/378] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=da668845afcd4a95bf6de91627f699b4 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:20.773186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2866-b511-4665-9b3e-d2f18863d51b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.773512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2866-b511-4665-9b3e-d2f18863d51b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.773744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2866-b511-4665-9b3e-d2f18863d51b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.775576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 192/379] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:20.780699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.780997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.781252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.818621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:20.819884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:20.870197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fe77d-6e87-4402-b95f-896eb85e4d19 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:20.897910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:20.906375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:20.907036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:20.907379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:20.907712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] PATH_INFO: `/v3/groups/462e002b0e424135bafe2f66365ade3c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:20.913867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] RBAC: Authorizing `identity:get_group(group_id=462e002b0e424135bafe2f66365ade3c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:20.916603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:20.918688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 188/380] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/groups/462e002b0e424135bafe2f66365ade3c => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:20.924698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:20.931926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 193/381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:20.964460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:21.037324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fda3c1-a936-4210-a5fc-1c1ddaf86757 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:21.084066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:21.084915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:21.085227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:21.085525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:21.086295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] RBAC: Adding query filter params (group.id=462e002b0e424135bafe2f66365ade3c, role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=5aa4b373e99e4d74b2148fd834c1abbb) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:21.086601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:21.088117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:21.111656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 189/382] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/role_assignments?group.id=462e002b0e424135bafe2f66365ade3c&scope.project.id=5aa4b373e99e4d74b2148fd834c1abbb&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:21.135817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:21.155215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ab4cd-232c-49f3-aea4-25bf63ef4e5b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:21.193992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:21.194624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:21.194905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:21.195203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:21.208167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:21.213265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:21.214199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:21.215031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 194/383] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:21.222443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:21.291462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:21.338760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ad3d3-e803-4a2c-b489-932ca50d27d6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:21.381513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:21.382128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:21.382403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:21.382669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:21.383376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:21.383674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:21.385036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:21.394694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 190/384] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:21.400063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:21.548363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc806da-32fe-4e0d-a9c0-b7dbae399420 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:21.594608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:21.595284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:21.595586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:21.595860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:21.599775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:21.601329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:21.603153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 195/385] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:21 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:21.609109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:21.783921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8d12d-7506-4b91-8612-34d4f0afbafa None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:21.818860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:21.848457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:21.849151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:21.849429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:21.849698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:21.850912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:21.852383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fb0e5-fe49-4479-9ecb-532f1ea70a84 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:21.854023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 191/386] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:21 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:21.862500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:21.897806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:22.047499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1126b-8a86-47f2-9b2b-37e8822ddc79 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:22.113074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:22.114185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:22.114666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:22.115197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:22.118877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, group_id=c2ff7eb1ef334a0b931c715b17f2a5d4, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:22.120960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aa7e7-7935-4174-89aa-3fd66e4e7f02 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:22.142580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 196/387] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 9 17:43:21 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:43:22.324666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b704-6103-49f3-a527-c63b460ca35d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:22.325441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b704-6103-49f3-a527-c63b460ca35d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:22.325441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25b704-6103-49f3-a527-c63b460ca35d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:22.326549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 192/388] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:22.333594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:22.334015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:22.334386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:22.354651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:22.356321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:22.511882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 197/389] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:22.540037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25b704-6103-49f3-a527-c63b460ca35d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:22.672387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25b704-6103-49f3-a527-c63b460ca35d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:22.711569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:22.712213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:22.712497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:22.712758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:22.725885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:22.730721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:22.733755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:22.737586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 193/390] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:22.740064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:22.743841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:22.905379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc467-26ab-48d6-a29f-1579a4556b14 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:22.965182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:22.965711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:22.965988 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:22.966250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:22.966918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:22.967244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:22.968607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7753a-8d1b-4a10-bce4-aefd48710cfe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:22.978616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 198/391] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:22.988050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.091060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:23.131361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c42686-aeb1-4848-adc4-84530c834894 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.132018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c42686-aeb1-4848-adc4-84530c834894 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.132498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c42686-aeb1-4848-adc4-84530c834894 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.134356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 199/392] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:23.139704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.140246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.140706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.160035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:23.161646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:23.174163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa819eec-8e03-4c13-9f39-4ec7f5601299 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:23.240238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:23.242216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.243245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.244463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.248210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:23.250272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:23.253660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 194/393] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:22 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:23.258236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.266306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:23.301811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 200/394] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:23.350820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.440580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb976b7c-e292-43ce-991d-8b2089a86b0a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:23.471353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:23.510358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c92468-c836-4702-80d3-1a37a3844c36 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:23.543696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:23.544342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.544623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.544891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.546037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:23.547501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:23.549463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 195/395] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:23 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:23.550055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:23.551278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.551753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.552224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.555000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.563981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:23.569830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:23.570737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:23.571587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:23.574229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 201/396] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:23.580698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.640250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:23.722805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7133-1a61-4f33-8e3e-c09e16d3bc3b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:23.740032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9b1c-ec81-4b77-8470-a0d278eafa81 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:23.764322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:23.764842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.765116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.765384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.766100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:23.766404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:23.767831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06b561-67ad-4f91-8add-5bb1bd088df8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:23.776955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 202/397] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:23.778238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1877f-f903-4e9e-abde-d53e7670e956 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.778519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1877f-f903-4e9e-abde-d53e7670e956 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.778788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1877f-f903-4e9e-abde-d53e7670e956 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.784232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 203/398] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:23.784357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:23.784964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.785088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1877f-f903-4e9e-abde-d53e7670e956 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.785242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.785511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.787754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] RBAC: Authorizing `identity:create_grant(project_id=7713e09455654744919d00b7f24e4f3e, group_id=c2ff7eb1ef334a0b931c715b17f2a5d4, role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:23.789226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7454eba-f80c-4b79-a877-862f4185eb3b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:23.804167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 196/399] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 9 17:43:23 2022] PUT /identity/v3/projects/7713e09455654744919d00b7f24e4f3e/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/da668845afcd4a95bf6de91627f699b4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:43:23.805817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:23.806102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:23.806516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:23.826674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:23.827954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:23.873759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:23.914810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 197/400] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:23.960729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:23.983539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1877f-f903-4e9e-abde-d53e7670e956 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.002187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.026745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.027350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.027739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.028016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.029183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.030785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.032942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 204/401] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:23 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:24.038926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.115426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ca22-2bd5-4dfa-99f1-b53664fe3395 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.180539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c91de-7ad7-48cf-82bd-937a66231f92 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.205547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.205547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.205547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.205547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.231967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-04f27e82-c2e8-4ae6-9d58-b928f2bbe683 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:24.239057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:24.240111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:24.240943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 198/402] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:24.241899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662426ba-a226-48b9-9241-a781bc9cdbab None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.247013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662426ba-a226-48b9-9241-a781bc9cdbab None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.247013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662426ba-a226-48b9-9241-a781bc9cdbab None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.247013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 199/403] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:24.247013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662426ba-a226-48b9-9241-a781bc9cdbab None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.253072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.254724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.255039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.255326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.256469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.257900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480465d-08c7-43e2-985c-0572b079f6fa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.260406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 205/404] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:24.262093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.262795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.263121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.286001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.287929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:24.289145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:24.355633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.376474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662426ba-a226-48b9-9241-a781bc9cdbab None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.393461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 206/405] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:24.393461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.432134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.432756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.433036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.433331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.434411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:24.434720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.436391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.452797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 200/406] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:24.452797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.535891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84e410-0df5-498c-8063-e91fcd3fa18c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.575909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.582943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.584292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.584397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f5185-22c8-42bf-bf34-8800e81752b4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.584879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.585336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.588582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, group_id=c2ff7eb1ef334a0b931c715b17f2a5d4, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.590236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.608789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 207/407] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 9 17:43:24 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:24.610724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.630486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.631023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.631358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.631636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.644921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:24.650807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:24.651945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:24.652767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:24.657128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 201/408] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:24.659158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.679633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.762782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038665e-130f-4a4a-bf89-180a96aea0d0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.781830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9039b-4c49-4432-91ce-1a5170d0bded None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:24.822498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.823942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.824756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.825234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.826883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.829065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.831908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 208/409] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:24.840703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.866127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:24.866754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:24.867132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:24.867410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:24.868101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:24.868406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:24.869815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:24.883042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 202/410] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:24.889298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:24.913103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:24.982056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ada34-cf6f-40cf-80ca-53ffb3fdb24b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.005748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870354d-3bda-4172-a129-4c8404d8f5c9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.007354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:25.024773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.025881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.026363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.026833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.028222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.030052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19123d20-a5ca-4d27-ae33-1aaaf2f08801 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.032482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 209/411] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:25.033959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a630f-ceb1-4984-aec8-d5b7c26eb024 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.034450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a630f-ceb1-4984-aec8-d5b7c26eb024 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.034940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a630f-ceb1-4984-aec8-d5b7c26eb024 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.036642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 210/412] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:25.042497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a630f-ceb1-4984-aec8-d5b7c26eb024 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.050327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.051024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.051353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.051624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] PATH_INFO: `/v3/groups/462e002b0e424135bafe2f66365ade3c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.052903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] RBAC: Authorizing `identity:get_group(group_id=462e002b0e424135bafe2f66365ade3c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.055342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4dcdb-5d4e-4a24-86ae-8c3a93030b89 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.059716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 203/413] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:24 2022] GET /identity/v3/groups/462e002b0e424135bafe2f66365ade3c => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:25.065489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.065489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.065489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.089787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:25.090898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:25.163086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 204/414] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:25.163963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.175311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a630f-ceb1-4984-aec8-d5b7c26eb024 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.179545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:25.215762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.216803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.217286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.217849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.221046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf9fcf6a00469089b1568b80c202cc, group_id=c2ff7eb1ef334a0b931c715b17f2a5d4, role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.222814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.239702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 211/415] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 9 17:43:25 2022] PUT /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc/groups/c2ff7eb1ef334a0b931c715b17f2a5d4/roles/da668845afcd4a95bf6de91627f699b4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:25.240991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.296552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c82ad-3307-4c78-831f-da68831b90fd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.299403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:25.362344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.363070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.363348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.363612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] PATH_INFO: `/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.365473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] RBAC: Authorizing `identity:get_project(project_id=5aa4b373e99e4d74b2148fd834c1abbb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.366946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.370786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 205/416] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:25 2022] GET /identity/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:25.375100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.423568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515274ff-99aa-49ec-bb1b-ffb91b97bcea None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.465041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.465696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.465970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.466249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.476510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:25.480615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:25.487117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:25.488096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:25.488886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 212/417] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:25.494282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.506448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884a83e-eb54-4a3f-9c10-ddfb17742000 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.587721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.588554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.588666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.588930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] PATH_INFO: `/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb/groups/462e002b0e424135bafe2f66365ade3c/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.596383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] RBAC: Authorizing `identity:create_grant(project_id=5aa4b373e99e4d74b2148fd834c1abbb, group_id=462e002b0e424135bafe2f66365ade3c, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.598080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.621589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 206/418] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 9 17:43:25 2022] PUT /identity/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb/groups/462e002b0e424135bafe2f66365ade3c/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:43:25.653104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd9aec-3c45-4351-86d1-a485b23df96a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.671483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:25.728347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.728972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.729254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.729526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.730268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:25.730572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:25.732344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:25.747407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 213/419] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:25.753755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:25.885628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513df79-50cb-4902-b3ef-9389f6d418fa None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:25.924265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:25.924878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:25.925154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:25.925427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:25.940080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:26.009101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:26.010720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:26.157550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 207/420] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:26.163196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:26.281012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffef7a9-d3ae-4ac7-aaa3-f857e88de27a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:26.316132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:26.316851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.317197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.317536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.318907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:26.320461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:26.322476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 214/421] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:26 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:26.327558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:26.392491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:26.463821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd01a3-ae60-4cab-a679-b273ca8c91e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:26.498901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:26.499540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.499822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.500096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.501216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:26.502785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8c1f1-dc48-462a-be30-443c4a02113d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:26.504688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 208/422] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:26 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:26.511849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:26.534338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9da-9fb2-43e3-9f04-4acd4684e516 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.534653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9da-9fb2-43e3-9f04-4acd4684e516 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.534874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe9da-9fb2-43e3-9f04-4acd4684e516 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.536489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 209/423] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:26.541546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.541827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.542094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.554873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:26.562954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:26.564199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:26.670038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277d4e3-cc66-406b-a934-c8eda68f0cef None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:26.701827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 210/424] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:26.731473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:26.731998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.732270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.732539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.733307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=7713e09455654744919d00b7f24e4f3e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:26.733612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:26.735060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:26.745882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 215/425] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:26 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:26.765818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:26.812427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:26.878993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696289f-ba71-4a0c-b9b2-916ed5724a6d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:26.926966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:26.927625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:26.927900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:26.928168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:26.941789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-74da5bf4-ef4a-4eab-8896-9cbe93fd1a25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:26.946522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:26.947429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:26.948578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 211/426] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:26.953941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.003712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a7d9-344d-4936-9ba3-bae196accd8b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.003990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a7d9-344d-4936-9ba3-bae196accd8b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.004161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a7d9-344d-4936-9ba3-bae196accd8b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.005789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 212/427] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:27.018310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.018621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.018853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.054615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:27.055835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:27.103980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1f2d1-c142-410d-b432-86b3510cb010 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.107347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.148035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.148392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 213/428] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:27.148661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.148941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.149216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.149893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:27.150198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.151613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.162899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 216/429] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:27.168565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.171982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.204561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.283503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a18dbe-1b87-451c-bfcf-e29d75455546 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.283939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249d88-e708-4e06-9b8a-e2b741c91d8e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.322339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.322917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.323588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.323805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.323901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.324133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.324546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.325043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.339093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:27.344385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:27.345283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:27.346103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 217/430] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:27.352425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.373552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.412018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.413951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.448933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.476279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e93c-a709-425f-83cc-7b325d16149a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.516615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.517752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.518340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.518812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.519695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] RBAC: Adding query filter params (name=member) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:27.520214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.521919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.531511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 218/431] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:27.539087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 214/432] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:27.540676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.543263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.639163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.670019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c72c8-7f29-4feb-b98c-678b442dd02c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.679052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f688c30-1a38-47d4-970f-89f2e36ffdd5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.716392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.717465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.717620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.717730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.718004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.726769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.727646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.728069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.728611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] PATH_INFO: `/v3/projects/7713e09455654744919d00b7f24e4f3e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.729887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] RBAC: Authorizing `identity:get_project(project_id=7713e09455654744919d00b7f24e4f3e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.731863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2719a498-85a1-48d1-ba2c-d6452dde47a5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.734094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 215/433] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/projects/7713e09455654744919d00b7f24e4f3e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:27.735526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc675d-d8fc-4258-8fb1-ea68d05ac007 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.736556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc675d-d8fc-4258-8fb1-ea68d05ac007 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.736831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc675d-d8fc-4258-8fb1-ea68d05ac007 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.738290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 216/434] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:27.741598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc675d-d8fc-4258-8fb1-ea68d05ac007 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.807937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.809456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.864095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc675d-d8fc-4258-8fb1-ea68d05ac007 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:27.891159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:27.913551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:27.914180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.914475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.914846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.915881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:27.917419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdb5c7-fc3e-42f0-ad1c-1cb4adb87513 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:27.927639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 217/435] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:27.927639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:27.927639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:27.927639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:27.928259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 219/436] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:27.933664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:27.943498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:27.944741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:28.012226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.030998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 218/437] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:28.030998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.075818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b87cf-3b0b-41d4-a799-9853b399e79a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.113994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.114587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.114860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.115170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.115884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=7713e09455654744919d00b7f24e4f3e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:28.116184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.117701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.131348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 220/438] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:27 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=7713e09455654744919d00b7f24e4f3e&role.id=da668845afcd4a95bf6de91627f699b4 => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:28.132149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.149576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfa07e-b5c4-4b8a-bebf-bfb202e32a76 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.191084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.195611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.196260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.196600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.196875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.198189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.199870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.206055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 219/439] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:28.207410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.288051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af39d3-f296-4550-b7a9-401d6d544fc4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.290415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.332755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.333300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.333578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.333846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.336481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a60af1-84b4-4443-af66-5148543b82c1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.349921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-41b9ca26-5330-48f5-93a0-eb2ccadb7de3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:28.356486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:28.357399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:28.358188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 221/440] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:28.364279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638053b-240e-4bab-89fb-0e43cc068e5f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.364567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638053b-240e-4bab-89fb-0e43cc068e5f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.364848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638053b-240e-4bab-89fb-0e43cc068e5f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.366132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 222/441] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:28.366819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638053b-240e-4bab-89fb-0e43cc068e5f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.374839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.375744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.376343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.376459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.377543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.379178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978ec4d-2c71-428d-b2b4-a6a6d7f3d041 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.385862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 220/442] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:28.388226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.388226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.388226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.406573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:28.408453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:28.495021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.496797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638053b-240e-4bab-89fb-0e43cc068e5f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.499112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 221/443] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:28.499112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.534485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.535108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.535385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.535695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.536350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:28.536645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.538039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.546371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 223/444] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:28.547044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.561420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.600106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d42b3-27e5-4066-9223-4135e2c96aa7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.635561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.636154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.636429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.636698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.637441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=b424450c766b4e32b0c1dbb8d5088e4d, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:28.637743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.639193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.645669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 222/445] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=b424450c766b4e32b0c1dbb8d5088e4d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:28.646472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.666990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86bc82-b8f5-4209-a045-6fb3e6a25910 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.706008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.706648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.706925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.707234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.719145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:28.724480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:28.725412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:28.726199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 224/446] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:28.732034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:28.758126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.768758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c747b79-f13d-460c-8bce-94579d2420f7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.796917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:28.808665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.809280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.809560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.809832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.851796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b46db-9f81-4946-93dc-ced1fdb232b1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:28.890329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.891964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.893505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:28.894134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:28.894423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:28.894670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:28.895361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:28.895659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:28.897009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:28.906293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 225/447] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:28.911717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.015261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 223/448] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:29.020907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.034448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.049643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.050795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7e8b3-9c27-4c0f-926a-3370104c3493 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.088712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.089322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.089603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.089879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.125837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd401f80-64a9-412c-9f9a-8d5281e2db08 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.160220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.161128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.161684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.161778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.162018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.162292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] PATH_INFO: `/v3/projects/8eaf9fcf6a00469089b1568b80c202cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.163442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] RBAC: Authorizing `identity:get_project(project_id=8eaf9fcf6a00469089b1568b80c202cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.164930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.166548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 224/449] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/projects/8eaf9fcf6a00469089b1568b80c202cc => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:29.175811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.254534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.271056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 226/450] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:43:29.276734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.277223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.281208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b2428-152e-4b67-ac43-79c9910a7529 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.317030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.317656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.317930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.318202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] PATH_INFO: `/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.319528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] RBAC: Authorizing `identity:get_group(group_id=c2ff7eb1ef334a0b931c715b17f2a5d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.320914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.322485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 225/451] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/groups/c2ff7eb1ef334a0b931c715b17f2a5d4 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:29.328143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.393887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bac619-9d1d-46a2-bcc1-a9cc19fea57e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.433421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.434055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.434330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.434602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] PATH_INFO: `/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.435772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] RBAC: Authorizing `identity:get_project(project_id=5aa4b373e99e4d74b2148fd834c1abbb)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.437240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.438903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 227/452] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/projects/5aa4b373e99e4d74b2148fd834c1abbb => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:29.444502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.463313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0d70b-e100-4dc6-9f40-e7b459aa4306 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.502625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.505770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.506430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.506643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.506917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.507744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] RBAC: Adding query filter params (group.id=c2ff7eb1ef334a0b931c715b17f2a5d4, role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=8eaf9fcf6a00469089b1568b80c202cc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:29.508050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.509482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.515806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 226/453] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/role_assignments?group.id=c2ff7eb1ef334a0b931c715b17f2a5d4&scope.project.id=8eaf9fcf6a00469089b1568b80c202cc&role.id=da668845afcd4a95bf6de91627f699b4 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:29.560563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a53bb9-83ff-4208-bd57-b016c62f1a0b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.602685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.603374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.603662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.603934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] PATH_INFO: `/v3/groups/462e002b0e424135bafe2f66365ade3c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.605189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] RBAC: Authorizing `identity:get_group(group_id=462e002b0e424135bafe2f66365ade3c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.606917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4745454-5aa5-4822-91d0-111fb60960b3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.608758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 228/454] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/groups/462e002b0e424135bafe2f66365ade3c => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:29.615884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:29.647739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:29.740484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144042b2-2b5c-455d-8450-a2e263d4df5d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:29.777280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:29.779042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:29.779042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:29.779042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:29.779328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] RBAC: Adding query filter params (group.id=462e002b0e424135bafe2f66365ade3c, role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=5aa4b373e99e4d74b2148fd834c1abbb) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:29.779482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:29.780917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec0f36-e689-4e80-b276-b0bcc934cb8c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:29.788188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 227/455] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 9 17:43:29 2022] GET /identity/v3/role_assignments?group.id=462e002b0e424135bafe2f66365ade3c&scope.project.id=5aa4b373e99e4d74b2148fd834c1abbb&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:32.465075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41688-b933-493a-8fab-1e22b20aca74 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.465591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41688-b933-493a-8fab-1e22b20aca74 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.465945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f41688-b933-493a-8fab-1e22b20aca74 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:32.467824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 229/456] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:32.472833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.473644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.474111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:32.499993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:32.502223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:32.576692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe5bb1-7508-41af-9811-8e138d82caa4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.577302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe5bb1-7508-41af-9811-8e138d82caa4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.577890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe5bb1-7508-41af-9811-8e138d82caa4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:32.580435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 230/457] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:32.585932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.586500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.586963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:32.597501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 228/458] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:32.612484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:32.613679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:32.620963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:32.644684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:32.732876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02b8d0-deb2-4c3c-88f0-cbf15c7b0433 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:32.773580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:32.774206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.774480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.774747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:32.781884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 231/459] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:32.795163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:32.796637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:32.815465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:32.816446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:32.817910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:32.817910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:32.817910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:32.817910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 229/460] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:32.822848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:32.827048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:32.913821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:32.917275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:32.953434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4c823-bb7e-47da-a2b7-e5bc75d82433 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:32.992572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:32.992996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947979a-86f6-4120-a228-f303cd4678f1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:32.993651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:32.994119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:32.994535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.006251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.008451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-081c8b16-b521-41c4-8c12-8cba5474f8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:33.029384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:33.030384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:33.031450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:33.031450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.031450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:33.031450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 230/461] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:33.032801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7418af-7158-4525-ad08-c0ce4c33d5e2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.033160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7418af-7158-4525-ad08-c0ce4c33d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.033497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7418af-7158-4525-ad08-c0ce4c33d5e2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.035239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 231/462] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:33.036451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7418af-7158-4525-ad08-c0ce4c33d5e2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.036755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.037521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.037958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.038365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.039280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:33.040097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.042627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159077-2e1b-4fb7-9dea-1d1194a4d0c2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.058095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 232/463] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:33.060278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.060831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.061258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.088528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:33.089890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:33.174400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 233/464] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:33.175538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.181096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7418af-7158-4525-ad08-c0ce4c33d5e2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.222906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.231871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.237925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.238532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.238809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.239108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.239753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:33.240054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.241423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.254133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 232/465] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:33.255054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.299626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5bb3e-bd7a-41a8-889c-532869abe96e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.333743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.334344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.334619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.334891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.335661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.337099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.361767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 234/466] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:33.364629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d1b3-58f1-4a8b-8efb-37e5ce67fd3f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.372040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.408199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.409093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.409229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.409494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.422777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.424692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6f0012af-20fb-42e8-be9b-f6fa1d4c4a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:33.440833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:33.441803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:33.443089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:33.443089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.443089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:33.443089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 233/467] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:33.448651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa206d5-6259-4cff-9ae5-8a3e85521e5e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.449139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa206d5-6259-4cff-9ae5-8a3e85521e5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.449362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa206d5-6259-4cff-9ae5-8a3e85521e5e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.451426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 234/468] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:33.452573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa206d5-6259-4cff-9ae5-8a3e85521e5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.505471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.528141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.532148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f1bf2-bbf0-41b8-855d-860c72741d7e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.586709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.587349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.587627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.588623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.589042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.590818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816df6-9dd8-49f8-966a-819529b5f01b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.604511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa206d5-6259-4cff-9ae5-8a3e85521e5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.615586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 235/469] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/users => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:33.617331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.617775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.618045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.639415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:33.640958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:33.650783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.651486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.651761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.652080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.653385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:33.653748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.655288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.671475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 235/470] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:33.677185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.728631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 236/471] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:33.762221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:33.785306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.791236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fdfe5-5ef9-4f0d-b852-c896f71ef1fb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.827469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.828232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.828600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.828963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.830230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.831846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd92d7-2451-46e7-86fa-1271c7411d3c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.857207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 236/472] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:43:33.897439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:33.918779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c11fd6-d528-4095-98b8-e3aad33d2a2d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:33.959223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:33.959880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.960163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.960440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.972217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:33.973705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:33.983672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650e1f-2b6f-4605-a11c-bcd3c10de8f0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.984013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650e1f-2b6f-4605-a11c-bcd3c10de8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.984217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650e1f-2b6f-4605-a11c-bcd3c10de8f0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.985853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 237/473] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:33.990446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:33.991392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:33.992634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:33.992634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:33.992634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:33.992795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:33.992795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:33.992795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:33.992984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 237/474] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:34.000229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:34.020099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:34.021647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:34.120146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 238/475] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:34.144171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:34.190218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea400be-8576-4ef9-ba9a-8d63a69352ac None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:34.243983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:34.272755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788753db-9c11-4705-a1e0-055ff21bae9f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:34.279445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84660127-117a-457d-88c1-af72e203b798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:34.280395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84660127-117a-457d-88c1-af72e203b798 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:34.280861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84660127-117a-457d-88c1-af72e203b798 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:34.281282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84660127-117a-457d-88c1-af72e203b798 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:34.283079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84660127-117a-457d-88c1-af72e203b798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:34.283431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84660127-117a-457d-88c1-af72e203b798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:34.284828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84660127-117a-457d-88c1-af72e203b798 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:34.286869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:34.309867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 238/476] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:34.320688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84660127-117a-457d-88c1-af72e203b798 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:34.339064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:34.339064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:34.339064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:34.339064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:34.339481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:34.339481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:34.347513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.sql.core [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:43:34.347513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 17:43:34.347513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [parameters: {'id': '299793213c7b449baaefbd3566a6779c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 17:43:34.347513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80213) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Aug 09 17:43:34.368559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:43:34.369482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application yield resource Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise self.value Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application session.commit() Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:43:34.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:43:34.371802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:43:34.372689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:43:34.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [parameters: {'id': '299793213c7b449baaefbd3566a6779c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:34.374273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 17:43:34.380090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.382093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 239/477] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 9 17:43:34 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:34.382093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:34.466379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84660127-117a-457d-88c1-af72e203b798 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:34.538923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:34.539557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:34.539745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:34.540015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:34.540670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:34.542112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:34.562368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c8768-60a2-44e8-9333-c1826b2fd863 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:34.603270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 239/478] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:34 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:43:34.629263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:34.629912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:34.630184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:34.630450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:34.642665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-834a17b1-554c-41c8-81be-0a665d606868 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:34.648559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:34.649505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:34.650427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 240/479] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:43:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:34.656302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:34.733025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:34.805720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd331a-5506-4891-9cac-271bb6b10194 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:34.844057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:34.844831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:34.845710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:34.846016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:34.846686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:34.846996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:34.848367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc2fdb-92ad-4c4d-9a1a-0346f51d8ba6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:34.858361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 240/480] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:36.148928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe153e8-9b1b-4470-8433-506a999bf310 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.149572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe153e8-9b1b-4470-8433-506a999bf310 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.149572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe153e8-9b1b-4470-8433-506a999bf310 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.151076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 241/481] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:36.156191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.156688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.156857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.176183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:36.177350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:36.215114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742655df-b8f9-4f85-a1e4-2a5057d076b4 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.215473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742655df-b8f9-4f85-a1e4-2a5057d076b4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.215715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742655df-b8f9-4f85-a1e4-2a5057d076b4 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.217297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 242/482] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:36.225064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.225552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.226032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.250051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:36.264229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:36.265858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:36.281036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 241/483] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:36.383027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.384596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 243/484] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:36.448441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:36.496446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.522366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfad147-5680-4787-9b90-9d1b440d5fbf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:36.577394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:36.578436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.578913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.579970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.606272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0c454829-5d77-4766-b911-d12744bbb873 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:36.610938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:36.611804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:36.612590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 242/485] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:36.615289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d039fa-ce33-4948-94f3-fb5e399c7e73 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.615857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d039fa-ce33-4948-94f3-fb5e399c7e73 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.616726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d039fa-ce33-4948-94f3-fb5e399c7e73 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.619070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 243/486] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:36.620190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d039fa-ce33-4948-94f3-fb5e399c7e73 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.650353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282caf99-837e-44bf-b5c8-6c156dd73e75 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:36.685772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:36.686512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.686691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.687010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.689786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:36.699592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c7d5e4e3-7467-44e8-a9c4-25e458bb09b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:36.705281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:36.706162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:36.706993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 244/487] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:36.712031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.712318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.712592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.734855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:36.736491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:36.743555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d039fa-ce33-4948-94f3-fb5e399c7e73 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:36.745507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:36.785423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:36.786481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.786961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.787460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.788429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:36.788976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:36.790648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:36.800410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 244/488] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:36.801610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.819088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 245/489] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:36.820588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.925644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959f-b9b4-413f-8aa7-8626ed0a1d2f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:36.938523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281064a6-c4d9-47ad-9579-98b064256037 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:36.968441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:36.969292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.969568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.969839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.970551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:36.970852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:36.971382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:36.972796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:36.987010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e6918-382e-46fa-b297-16203325992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:36.987010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6918-382e-46fa-b297-16203325992d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:36.987010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6918-382e-46fa-b297-16203325992d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:36.987010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6918-382e-46fa-b297-16203325992d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:36.987397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 245/490] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:36.988214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:36.998451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e6918-382e-46fa-b297-16203325992d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:36.999950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e6918-382e-46fa-b297-16203325992d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f32e6918-382e-46fa-b297-16203325992d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:37.016887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.018361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:37.019211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:37.019211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.019211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:37.019211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 246/491] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.019438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e6918-382e-46fa-b297-16203325992d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.028450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.119857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f506-2802-4eb1-b373-7b98df99a3de None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.157622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.158633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.159165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.159648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.159892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e6918-382e-46fa-b297-16203325992d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.170778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:37.176395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.177245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:37.178022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 246/492] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.182897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.205160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.205834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.206224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.206588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.217889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.219453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:37.234637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.235576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:37.237037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:37.237037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.237037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:37.237037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 247/493] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.237679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.279493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.299425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.317824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1722f-6f45-46b4-a40f-5e470256b4c4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.354260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.354965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.355263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.355531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.356427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:37.356725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.358215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.369989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d8a66-4002-4d92-9a87-c80bd845f9d1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.375563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 247/494] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:37.375563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.404385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.404985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.405267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.405535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.406203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:37.406508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.407916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129bf01-737d-442e-9377-aa02c3d9ce3f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.415761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 248/495] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:37.421502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a28ba-1cb9-41c9-9685-c53bcc2cd1fc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.423747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a28ba-1cb9-41c9-9685-c53bcc2cd1fc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.424033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a28ba-1cb9-41c9-9685-c53bcc2cd1fc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.425417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 249/496] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:37.426581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4cfd-66f8-439b-90b6-83589fc9c1e2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.426804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4cfd-66f8-439b-90b6-83589fc9c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.427102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef4cfd-66f8-439b-90b6-83589fc9c1e2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.428555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 250/497] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:37.428976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef4cfd-66f8-439b-90b6-83589fc9c1e2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.480165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d12186-2b60-4bd4-8bcd-51084738ea65 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.515151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.515751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.516029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.516301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.516962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:37.517266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.518628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.521879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.531186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 248/498] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:37.531985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.536272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef4cfd-66f8-439b-90b6-83589fc9c1e2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.543987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.571661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.572274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.572549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.572816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.579853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.581290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4eaaee10-cbd9-4b4f-87be-0d090fbdeb41 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:37.594360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.595876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:37.596836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 251/499] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.597391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76d0e-e837-4da8-be33-055a6813209f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.597391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76d0e-e837-4da8-be33-055a6813209f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.597699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa76d0e-e837-4da8-be33-055a6813209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.615940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3fa76d0e-e837-4da8-be33-055a6813209f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:37.617147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3fa76d0e-e837-4da8-be33-055a6813209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:37.639335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03e724-7bf9-400b-9847-35505a9711f9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.673747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.674360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.674637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.674906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.682703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 252/500] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:37.683964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.684246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.684515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.691100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.692557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:37.706586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.707537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:37.708378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:37.708378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:37.708378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:37.708667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:37.710119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:37.712516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 249/501] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.713379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.769943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.778000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 253/502] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:37.778831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.796432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:37.813961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153ba01-b73e-45d4-b111-a8f2d7c776c5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.847587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.848232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.848519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.848789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.859069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.860451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:37.873249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:37.874220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 250/503] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:37.875143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.884191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d379e-bed3-400a-8331-d31028bc1eb2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.919223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:37.919818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:37.920091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:37.920359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:37.921082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:37.921383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:37.922798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:37.960902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 254/504] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:37.961635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:37.981427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64c610-1ee9-4b41-be61-cb694da7966f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:37.999690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.023107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.024120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.024604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.025080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.037533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.039052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.046364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:38.056631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:38.057544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:38.058373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:38.058373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:38.058373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:38.058373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 251/505] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:38.059425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.091280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c144a4-3180-4932-bac5-3b58119d25ce None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.127380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.127984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.128263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.128534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.129181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:38.129547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.130938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.142547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 255/506] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:38.143371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.178534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27fde-25e9-4691-bae3-d42005c8be28 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.213353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.213880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.214158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.214434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.225332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:38.230393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:38.232487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:38.234082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 252/507] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:38.236193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.245922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26188bdc-b758-4775-a895-f667390c2f19 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.268094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.275356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.281325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.281919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.282201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.283055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.283822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:38.284121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.285544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.309085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 256/508] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:38.309877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.344528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591f0f8-0d41-4654-94d7-1c4174385e92 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.382249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.383148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.383434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.383789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.416851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7215-75c7-4d7c-98e2-db2e8e44e669 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.449695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.450283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.450553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.450817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.450907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.451475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:38.451770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.452324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.453112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.464933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 257/509] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:38.465645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.482511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.515474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.553458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 253/510] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:38.554406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.566594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03477dc-b74a-46f8-89d9-b263f981cb9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.599576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.600691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.600973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.601246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.609709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.611245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:38.622539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:38.623654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 258/511] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:38.624533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.651469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154778bd-5365-4c79-bc57-1335d2fc7c30 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.679371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.685730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.686311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.686591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.686857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.687547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:38.687844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.689814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.697584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 254/512] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:38.698335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.728816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183aba3-2d7d-4705-a2df-606db3472696 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.739119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.764781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.765383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.765665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.765935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.795083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda16bea-6cd0-4f35-aaa0-2fb2f2060ad2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:38.829504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:38.830095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:38.830368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:38.830634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:38.831286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.832718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.835303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:38.836780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:38.853289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 255/513] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:38 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:38.864040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.920389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.954387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 259/514] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:38.956732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:38.968318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:38.986530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aaa3e3-6315-4bc4-b8c2-5edfb2c43f6a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.023665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.024678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.024678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.024868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.036128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.037980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:39.055583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:39.056642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:39.057552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:39.057552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.057552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:39.058326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 256/515] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:39.059250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.094233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78078b-6f4e-4ce8-ad09-d1703f443c5d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.130096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.130767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.131080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.131354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.132071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:39.132373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.133790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.150553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 260/516] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:39.151395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.176019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.180053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a9a9d-69bf-4dab-a597-e4d5e7d866a7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.216878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.217496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.217877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.218540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.218908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.231731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.233335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:39.248910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:39.249833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:39.250663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:39.250663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.250663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:39.250663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 257/517] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:39.250663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.259548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424892af-71bd-4c66-9913-e3f70cef6a88 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.293872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.294517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.294770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.295087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.305707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.307204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:39.321927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:39.322827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:39.323735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:39.323735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:39.323735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:39.323735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 261/518] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:39.323953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.363071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877497d-12dd-433d-92fc-5e34b6915344 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.401428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.402144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.402424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.402694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.403368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:39.403668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.405022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.426065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 258/519] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:39.426065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.431971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a475d-dd47-4cf6-85ad-c40704479f29 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.445529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.451936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.472548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.473192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.473474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.473745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.541659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8648e-a6c4-401a-ba11-4afd9f34b50f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.551573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.553133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.577170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.577766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.578074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.578345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.579011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:39.579831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.581190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.592379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 259/520] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:39.593178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.663882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 262/521] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:39.664562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.683118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.684067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.715921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c9734-600c-4bdc-8010-4b6f2d0d0bfe None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.757532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.758130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.758403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.758741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.759430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:39.759733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.761100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.773783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 260/522] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:39.774662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.779928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016087-fe98-412f-bd11-8dcf2a20c2be None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.822242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.822927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.823343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.823737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.831269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.832786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:39.844331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:39.845943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:39.846835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 263/523] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:39.847407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.892584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70c693-c695-44f1-9263-cf5764f63d1c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.925359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.931074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.931702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:39.931888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:39.932007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:39.932238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:39.941066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:39.942492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:39.955519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:39.956595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 261/524] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:39.957627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:39.960831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aac804-5b59-4232-9cc6-dedd60233532 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:39.999302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:39.999920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.000195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.000465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.011703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.013167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:40.027685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:40.028817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:40.029658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:40.029658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:40.029658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:40.029658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 264/525] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:40.029970 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.082542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9c54d-319e-422b-a06b-a52bc8981ffd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.123376 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.123999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.124279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.124555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.133343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.135037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:40.147837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:40.148765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:40.149666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:40.153962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 262/526] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:40.154688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2149e8-3815-4dd3-8f60-a08aad779d96 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.154922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.186831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.194628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.195412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.195758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.196091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.196824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.197130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.198575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.214018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.218511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 265/527] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:40.219312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.269883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52e8cc-1834-4615-bc4e-3aac67e82c8c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.307380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.307979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.308264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.308539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.309263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.309571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.311024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.325518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876fc5e-9615-4555-b2f7-4228d96418a2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.348540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 263/528] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:40.349383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.381645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.383401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.384341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.385517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.387256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.388236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.392304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.418736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 266/529] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:40.420582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.463244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a65c0e-537d-4ce6-b7f6-761d70ea5670 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.475787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.476279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.502584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.503313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.503588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.503851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.504536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.504836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.506282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.525146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 264/530] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:40.526279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.532779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01ed08-70e2-42c7-9105-291772f63e55 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.569950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.570471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.570745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.571048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.571755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=b46658fdbcde40498e87d37d79609c83) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.572059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.573494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.587010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 267/531] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/role_assignments?user.id=b46658fdbcde40498e87d37d79609c83&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:40.587010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.678551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c60156-d657-4251-81b5-2ffe5f58e8a6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.718297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617ecd-4f7e-4a51-ad34-cdd0ce40602a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.738256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.739230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.739537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.739669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.740084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.774914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.776175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.777071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.777798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.788747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.790495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.792487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:40.808308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:40.809174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:40.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:40.812414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 268/532] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:40.813362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.834476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:40.836246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:40.947666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962e81-6173-4074-809a-d2a258932d3b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:40.950844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 265/533] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:40.951347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:40.995861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:40.996784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:40.997221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:40.998134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:40.999047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:40.999587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.002138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.010674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 269/534] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:41.011734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.063118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.071307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.116016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff607b-494b-46ba-9477-435127bccb96 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.147106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018ee27-b62d-4250-8c04-d730cc82f73b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.154439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.155080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.155360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.155631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.156332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:41.156633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.158766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10029ecc-282c-4dd8-811d-3376c08bdef4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.171811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 266/535] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:41.173154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f615d-b046-4dd2-9350-3ac19ae71801 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.173438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f615d-b046-4dd2-9350-3ac19ae71801 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.173711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f615d-b046-4dd2-9350-3ac19ae71801 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.175115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 267/536] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:41.180012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f615d-b046-4dd2-9350-3ac19ae71801 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.197978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.198536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.198812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.199120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.215194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.217577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1c9d7c72-fc76-4ce8-aa17-5c243fe99354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:41.245483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:41.246428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:41.247339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:41.247339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:41.247339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:41.247339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 270/537] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:41.248420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.248706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.248975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.272406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:41.273746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:41.316772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f615d-b046-4dd2-9350-3ac19ae71801 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.355413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.361007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 271/538] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:41.362013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.362618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.362900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.363203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.363520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.363935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3098494139cc41359d5c34b2a8428c6e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:41.364243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.365949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.379021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 268/539] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/role_assignments?user.id=3098494139cc41359d5c34b2a8428c6e&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:41.390409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.407402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.523579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4942e7-09ea-40b9-b64f-a1235f23c626 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.569107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.569754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.570037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.570311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.593133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9798e1-2bc0-4e2d-b0e5-f4ce50b190b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.597098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:41.611712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:41.612676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:41.613915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 269/540] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:41.619945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.651266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.652261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.652927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.654244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.655293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:41.656635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.659121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.684899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 272/541] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:41.694921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.718272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.769275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159a21-b8b0-4fe1-9401-476abd0fb78d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.808014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.808628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.808949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.809437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.810163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:41.810470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.811912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.821348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 270/542] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:41.826387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:41.867061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:41.913637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31183c63-e51a-436d-9470-0a2f637c8ab7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:41.952141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:41.953047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:41.953471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:41.953925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:41.962412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:41.964327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:41.977503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:41.978559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:41.979636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:41.982071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 273/543] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:41.988024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:42.078614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7240b-a42f-4e64-a047-e92f3e7e0ac9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.111769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:42.139755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.140751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.141175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.141614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.158963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.162284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:42.202408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b7e1-7fdb-4681-b3dc-b6bc7bed9613 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:42.205509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:42.206579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:42.207564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:42.207564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:42.207564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:42.210159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 271/544] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:42.216021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:42.265683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.266297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.266580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.267589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.268313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:42.268867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.270314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:42.283353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:42.285552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 274/545] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:42.297960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:42.424492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11420cb1-4891-4dc6-b205-0fe25a058796 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.448339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e30e3-30f8-452c-b525-dd3fc90c4e53 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.461181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.461872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.462225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.462564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.463363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:42.463741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.465180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:42.478024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 272/546] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:42.483381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:42.509621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.510220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.510500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.510771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.511554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=768dabebad264236b8f43dbd1c85a964) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:42.511863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.513365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:42.522038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 275/547] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:42 2022] GET /identity/v3/role_assignments?user.id=768dabebad264236b8f43dbd1c85a964&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:42.563315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:42.684533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4bae6-582b-402c-ae2e-12588ce58fd5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.734908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.736398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.736854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.737270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.748435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.750154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9f7a901b-f043-4115-9531-215c60cc243e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:42.767964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:42.769164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:42.770814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:42.773269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 273/548] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:42.778682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:42.867146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:42.917291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5eff1-3fe8-4508-9148-b9e5a9c31563 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:42.919092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7391b-1dbb-4f5a-bff0-729bc5d77914 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.919794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7391b-1dbb-4f5a-bff0-729bc5d77914 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.920229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7391b-1dbb-4f5a-bff0-729bc5d77914 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.927333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 274/549] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:42.934054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.934383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.934881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.964527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:42.965976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:42.966698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:42.967464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:42.973996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:42.976485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:42.977467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:42.978854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:42.980062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:43.019721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 276/550] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:43:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:43.022062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.112072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 275/551] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:43.145104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.183348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.198221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e146c-682d-4d4d-8b7a-c6dc1f769e43 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.243260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.256075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.257042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.257561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.258066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.339135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b1cd-8f8e-4ebc-937b-04d90572dfb7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.368657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:43.371781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:43.411725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.412813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.413262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.413697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.434091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f12520a3-f565-45b6-b8eb-87f69b346fac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:43.439406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:43.440415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:43.441221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 276/552] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:43.442014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec6e4-d91c-4462-94cb-2f121a0b569f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.447008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec6e4-d91c-4462-94cb-2f121a0b569f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.447008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec6e4-d91c-4462-94cb-2f121a0b569f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.448304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 277/553] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:43.449585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.449870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.450146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.481203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:43.482455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:43.533767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 277/554] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:43.537056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.566157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.571498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 278/555] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:43.572589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.585803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.682800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfca121-2e34-4534-8ca2-f7a8cbd671cd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.705944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1befb17c-f0c2-4ce3-940e-edfdf8d38552 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.730683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.732319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.733004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.733440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.734536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:43.735418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:43.737560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:43.743395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.744034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.744313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.744583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.749251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 278/556] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:43.755521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.758497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:43.760276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:43.782554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:43.783568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:43.784435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:43.784435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:43.784435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:43.784435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 279/557] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:43.784885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.863249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.866504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:43.885168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec131949-0053-4724-9772-7d0a252419f7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.913416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd0171-c89d-48eb-b2eb-06fffb1c7c8e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:43.924389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.925015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.925292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.925561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.936113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:43.941818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:43.943409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 279/558] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:43.944227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:43.957324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:43.957876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.958151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.958426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.969824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:43.971286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-19fe74de-feac-43e4-b8ca-3c2403036389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:43.986440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:43.987608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:43.988626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:43.988626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:43.988626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:43.988626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 280/559] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:43.989550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89fc4-8cda-4843-8d77-2bbe061c4e43 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:43.989837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89fc4-8cda-4843-8d77-2bbe061c4e43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:43.990307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d89fc4-8cda-4843-8d77-2bbe061c4e43 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:43.992737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 281/560] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:43.993313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89fc4-8cda-4843-8d77-2bbe061c4e43 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.074158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa1f03-f1fa-4d63-a88d-37e4b7d28bd8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.118465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.119665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.119665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.120185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.121086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:44.121517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.122935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.124018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d89fc4-8cda-4843-8d77-2bbe061c4e43 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.136089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 280/561] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:44.140083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.141456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.145873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.168744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.169231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.169504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.169769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.170416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:44.170711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.172108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b7914-5e49-49ef-a8dd-ccb49e1b803b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.186615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 282/562] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:44.187471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.187803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.187978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.207068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:44.208599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:44.264681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051051e-cbc2-4daa-be34-84e93a287de7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.285311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 283/563] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:44.286191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.313609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.315091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.316084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.316555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.317670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:44.318410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.320713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.334709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 281/564] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:44.339763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.405711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d4a5e-0c4f-4693-98ee-8ee4e1ec0f62 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.409477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.420065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.448333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.449187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.449460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.449801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.459801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.461521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.467561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff26d56-77f9-4601-bb3e-f462a4c7c5bf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:44.475323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:44.476412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:44.477497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:44.478245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 284/565] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:44.483327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.518037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.518665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.519021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.519300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.532970 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.534451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:44.549664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:44.551215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:44.552256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:44.552256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:44.552256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:44.552434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 282/566] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:44.553118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.617471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ccf83-bc3e-4af6-a60a-ee4f73e8e265 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.655246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.655879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.656145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.656409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.668034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:44.672628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:44.679691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:44.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:44.681132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea45-e40c-47a1-8a24-e6cf72dbb953 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.681209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 285/567] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:44.681209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.699972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.707559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.724080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.724612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.724887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.725231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.736073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.737549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:44.749646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:44.751745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 283/568] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:44.753625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.806359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fa20f-b63f-4795-b8be-1b2d5ce83f95 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.843506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.844106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.844385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.844652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.845899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:44.846198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.847686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.861843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 286/569] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:44.862661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.875568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee81216-08e6-4e3f-bdf1-b3e637b3352a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:44.915720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:44.916594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:44.916864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:44.917130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:44.917767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:44.918065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:44.919463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:44.935621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 284/570] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:44.935621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:44.951073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.969244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:44.977587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521bef8-8aa3-47a3-b660-c9302cf20c74 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.012223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.012826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.013103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.013372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.014046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.014379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.016137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.024178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 287/571] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.024953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.053864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db788883-3fc6-423a-92bf-6e0dc926761a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.100091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.101023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.101473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.101946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.103589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.103589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.107211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33abf5-cd9a-49dd-90e6-53ae8bed0a3d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.135370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 285/572] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.136616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508d09b-8d5d-4acf-bdec-193a50445299 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.136899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508d09b-8d5d-4acf-bdec-193a50445299 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.137218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508d09b-8d5d-4acf-bdec-193a50445299 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.138607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 286/573] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:45.139129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508d09b-8d5d-4acf-bdec-193a50445299 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.152692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76ac31-91c0-4b93-b2ef-69da1712ebe2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.186888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.187519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.187796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.188069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.188826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=eae392d38ade4665b2385bae93b3e688) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.189133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.190556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.195936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 288/574] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/role_assignments?user.id=eae392d38ade4665b2385bae93b3e688&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.196719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.211668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.249135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.260416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508d09b-8d5d-4acf-bdec-193a50445299 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.303320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.303961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.304301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.304575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.312492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.314015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.327554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a3b32-f843-453f-8478-508f4705f929 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:45.328540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:45.329480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:45.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 287/575] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:45.331040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.370760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.371420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.371728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.371998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.383551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.385078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-dea07370-feda-45dc-be82-b0ef8d5db756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:45.399797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:45.400862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:45.401949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:45.401949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:45.401949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:45.401949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 289/576] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:45.402224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.402305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.402641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.422390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:45.424324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:45.458803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efeffc-de9c-4911-a942-f0c7ccb769c9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.486754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.498300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.498926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.499283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.499499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.500125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 290/577] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:45.500856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.526137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.527642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:45.543190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:45.544690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:45.545589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:45.545589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:45.545589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:45.545589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 288/578] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:45.545589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.554992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.617595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd9a0b-7751-412f-a639-cf19a6ccd1b9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.659238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.660052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.660339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.660610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.661058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a1dec-8584-471a-94d7-bf54ebc92062 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.661342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.661648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.663112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.689427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 291/579] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.689427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.702760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.703396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.703684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.703959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.705156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.705468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.706850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.722234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 289/580] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.723117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.754406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.800908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:45.810171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec8669-ae3f-4d6e-9abe-fe29a522f305 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.840953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41f6c-f498-43ee-85c2-e1337c7952ae None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:45.851797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.852424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.852703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.852972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.865191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:45.870058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:45.871592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:45.872650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:45.877405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 292/581] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:45.878180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:45.894896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:45.895561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:45.895841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:45.896120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:45.896775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:45.897079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:45.898445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:45.917570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 290/582] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:45.917570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.000020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799559-053e-4d6b-a30a-cdd098dd1b0c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.012088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.038256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.038992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.039345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.039688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.039783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ab3ce-c8e1-4966-a5e0-b6498fa287a9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.051580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.053120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.073078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:46.076782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.077718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:46.078593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:46.078593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.078593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:46.078593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 293/583] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.078830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.091049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.091950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.092867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.093315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.102724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.104334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:46.115990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.117177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 291/584] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.123686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.198880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a8fce-de1f-4ac0-9add-a4e5f2881daf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.232371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b6080-00a2-48b5-b307-6830e5f1d688 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.237826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.238427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.238705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.238992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.239671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:46.239972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.241466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.257270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 294/585] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:46.257270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.273592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.274509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.274921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.275241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.282606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.284372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.284929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:46.296621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.298227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:46.299170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 292/586] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.299648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.359040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.377499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde481c-fcb8-4c2d-ac8d-9881eb291a81 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.417293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.418072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.418072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.418638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.419112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:46.419414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.420780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.432274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53993474-9685-478d-b542-4c6b2c5bb04d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.437935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 295/587] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:46.437935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.489459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.490059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.490338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.490608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.491362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:46.491660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.493073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.515289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 293/588] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:46.516289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.541904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.558013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad8a59-b93c-4e7a-bd8c-f2dea953d423 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.595633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.596255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.596525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.596793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.607463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.608894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:46.624480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.625449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:46.626440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:46.626440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.626440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:46.626440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 296/589] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.626440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.638857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176889d3-3fbd-4e09-aee3-49e3ace20278 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.641370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.682398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.683316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.683592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.684289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:46.684594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.686028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.697537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 294/590] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:46.698340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.745352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb1255-3d3f-432a-9a64-332f52478d46 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.780928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.781908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.782692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.782954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.785727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.791055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.792769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:46.804585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.805472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:46.806352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 297/591] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.806820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13040-a190-4305-8382-026f3c41840b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.806901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.842863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.843902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.845653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.846084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.858825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.861066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:46.876606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:46.877634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:46.878694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:46.878694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:46.878694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:46.880075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 295/592] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:46.880955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.900791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:46.920633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc35051-20d4-4cea-835c-261689ebfca6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:46.964138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:46.964750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:46.965149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:46.965421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:46.966065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:46.966367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:46.967937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:46.981387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 298/593] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:46.987260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:46.991370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff38c-2942-49b6-9360-cec0858ff057 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.034916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.035971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.036489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.036980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/b46658fdbcde40498e87d37d79609c83/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.061988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=b46658fdbcde40498e87d37d79609c83, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.064616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.065147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.091083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 296/594] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:46 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/b46658fdbcde40498e87d37d79609c83/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:47.092104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.137499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecb9da-8331-42fa-baf4-2d6b1245a26e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.182472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.195536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.196524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.196987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.197458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.199082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:47.199622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.202251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.222629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 299/595] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:47.223653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.237648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc21e9-dd85-4715-a882-6aac1843e6fd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.280641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.281253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.281535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.281806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.284268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:47.284268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.284710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.296359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 297/596] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:47.297150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.352570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3d2f7-7e7a-4742-add3-8d8c0fd53462 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.369412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.393007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.393636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.393937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.394208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.402545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.404033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.413878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c847933-a614-46c4-9fac-c7ecd1b5de05 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-832c42a2-148c-44f6-915d-65305efc8118 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:47.415339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:47.416237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:47.417149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 300/597] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:47.417716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cc09-987f-47cc-b9c0-fdec4b31cc6d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.418109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cc09-987f-47cc-b9c0-fdec4b31cc6d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.418199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10cc09-987f-47cc-b9c0-fdec4b31cc6d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.421469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 301/598] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:47.421469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10cc09-987f-47cc-b9c0-fdec4b31cc6d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.450818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.461986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.462646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.462927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.463230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.478599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=a136641c41464f1f969fe6c16dfe3501, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.480120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.503003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 298/599] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:47 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:47.503003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.547470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10cc09-987f-47cc-b9c0-fdec4b31cc6d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.593919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.594927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.595749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.596025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.607267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.608914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ef7a4a49-f55e-4063-9e6d-0e7b4afa5c34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:47.625550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:47.627065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:47.628620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 302/600] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:47.631011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.631011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.631011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.643260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316134b-42ba-4ea8-b3e7-c8561e461a50 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.654939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:47.657152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:47.693667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.694682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.694966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.695268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.696014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:47.696320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.698078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.706226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.719578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 299/601] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:47.720436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.762653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 303/602] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:47.764056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.767464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:47.858032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214cd8-7174-429a-bc84-b126366a4003 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.890157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1278ef-17f1-4d2c-9c11-ca10dcd4d180 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:47.896901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.897536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.897813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.898086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.898876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:47.899219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.900735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.914301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 300/603] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:47.915764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:47.937532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:47.940728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:47.940728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:47.940728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:47.956438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:47.958911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:47.979308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:47.980296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:47.981432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:47.981432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:47.981432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:47.986126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 304/604] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:47.987031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.015041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.061166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.072302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f285fa-7ca3-4b5a-a2e9-687ecbd3f91a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.121170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.121727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.122010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.122286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.124140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988401e-c428-49b0-985d-ef94a7692869 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.139271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:48.146070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:48.147077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:48.147923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:48.151112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 301/605] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:48.152172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.168330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.169360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.169895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.170366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3098494139cc41359d5c34b2a8428c6e/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.187224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=3098494139cc41359d5c34b2a8428c6e, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.189024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.210473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 305/606] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:47 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3098494139cc41359d5c34b2a8428c6e/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:48.211706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.304303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.369622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58c95-3e84-4cda-a5dd-9c35c11c90b0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.411993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.412525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.412801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.413069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.413247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ba7b2-6e2d-449d-8f0a-b950e88926b9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.413801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:48.414095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.415725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.431244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 302/607] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:48.437112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.455842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.488437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.489432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.489917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.490414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.491324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:48.492070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.493672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.502959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 306/608] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:48.509409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.613745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a2ef9-f1a6-4afc-9d67-4a5d648d1e2c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.616565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36342c-be00-4e60-b869-ef78eda41d97 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.647256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.654485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.655412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.655918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.656390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.668949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.670734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:48.687588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:48.688702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:48.689571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:48.689571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:48.689571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:48.691854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.692600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.692766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.693071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.694155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 307/609] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:48.700281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.708220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.709680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:48.733555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:48.734507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:48.735456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:48.741802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 303/610] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:48.743375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.819788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0139c-44fe-419a-98b8-d67c35fba65a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:48.856524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:48.857508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:48.858027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:48.858505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:48.859379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:48.859885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:48.861594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:48.874539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 308/611] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:48.880926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:48.911368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.956669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:48.966747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c50e6-4190-4f0b-b047-94baae17003a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.039901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873d56c-9ff2-4def-8a6d-65329c6fdf44 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.044102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.044710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.045052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.045317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.046073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:49.046373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.048471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:49.074745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 304/612] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:49.087022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.118861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.119897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.121179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.124376 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.136335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.138829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:49.159748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:49.160810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:49.161791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:49.172920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 309/613] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:49.180419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.259883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd57677-7e44-4e28-b9b4-a5e10dd41d49 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.295137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.295789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.296253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.296705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/768dabebad264236b8f43dbd1c85a964/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.312241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=768dabebad264236b8f43dbd1c85a964, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.314281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077aadb-3d3c-4c48-bde5-f6b131b6f7bc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:49.318609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:49.347984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 305/614] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:49 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/768dabebad264236b8f43dbd1c85a964/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:49.411180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132149cc-6f8d-4f9a-8d4c-38d6b9a4b79e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.413015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:49.438248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f595df4-730d-4297-81ac-79c20f68372c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.439121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f595df4-730d-4297-81ac-79c20f68372c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.439366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f595df4-730d-4297-81ac-79c20f68372c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.441017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 306/615] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:49.446549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.446875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.447188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.464484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.465160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.465437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.465702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.466753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:49.467142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.468598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:49.487810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:49.489032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:49.498305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 310/616] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:43:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:49.504006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.617205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e306754-3e8f-4f95-a250-994c3bf2328f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.624557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 307/617] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:49.656111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.660887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.661526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.661802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.662069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.673904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.675457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:49.692593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:49.694763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:49.696832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:49.696832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:49.696832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:49.703016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 311/618] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:49.706797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.740677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:49.837776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29769d6-a7bf-462b-9faf-d3671475e2dc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.856530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:49.903309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.904390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.904880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.905668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.920882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:49.927296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:49.928245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:49.929265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:49.932184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 308/619] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:49.937182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd17567-898f-4f0a-86bb-2e8fc9e012a4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:49.938516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:49.988259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:49.989266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:49.989755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:49.990782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:49.991773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:49.992324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:49.994830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f60a19-e998-4296-8f03-dbf278c890e2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.012985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 312/620] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:50.014900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef2e18-4bd8-48cb-bc73-74fde9f948af None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.015840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef2e18-4bd8-48cb-bc73-74fde9f948af None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.017155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef2e18-4bd8-48cb-bc73-74fde9f948af None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.019260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 313/621] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:50.020244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef2e18-4bd8-48cb-bc73-74fde9f948af None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.069907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6d2f9-576e-4458-9d3b-b457d7932d0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.115665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.122919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.123873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.124787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.125317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.126830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:50.127413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.129230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7dce3-9676-46d1-9310-e3bec13154cb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.146689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 309/622] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:50.146689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.146689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.146689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.155831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef2e18-4bd8-48cb-bc73-74fde9f948af None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.171394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:50.174163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:50.195387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.195949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.196234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.196502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.204079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.205525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.213679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:50.218531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:50.219554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:50.220449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:50.223052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 314/623] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:50.224159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.270378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 310/624] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:50.279245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.357157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3b22-4af1-48ae-9d1f-c8f3dd796f0d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.388679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.395162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.396030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.396326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.396602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.408740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.410163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.415142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1c3f0-a290-457a-81e8-2ae80a2c97c6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:50.426295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:50.427324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:50.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:50.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:50.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:50.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 315/625] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:50.428965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.461993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.462903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.463436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.463899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.464611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:50.464919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.466395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.487343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 311/626] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:50.489512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.521611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.563599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7642f3-fc8d-4c47-aefe-7c693a5a839a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.602684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.603330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.603662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.603974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.606002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1174f4-c0e3-4697-90c2-48ba513cbd38 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.616839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:50.622034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:50.628215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 316/627] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:50.629053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.646172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.646860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.647182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.647459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.648114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:50.648417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.649775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b6d0e-03da-4bbd-a1c6-ff80bf0778d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.662407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 312/628] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:50.663583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e73f62-e16e-410e-96bf-629a4215593a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.663868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e73f62-e16e-410e-96bf-629a4215593a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.664139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e73f62-e16e-410e-96bf-629a4215593a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.667022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 313/629] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:50.667022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e73f62-e16e-410e-96bf-629a4215593a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.682109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.745908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877c94-6cb4-46f2-b243-366098fb643d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.764496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:50.784935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e73f62-e16e-410e-96bf-629a4215593a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:50.794311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.795176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.795933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.796331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/eae392d38ade4665b2385bae93b3e688/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.814407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=eae392d38ade4665b2385bae93b3e688, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.815876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf70ac2-e198-4a24-b48d-98a67a06279f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.823706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:50.824371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.824643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.824928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.825601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:50.825874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:50.827250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:50.835139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 317/630] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:43:50 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/eae392d38ade4665b2385bae93b3e688/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:43:50.843475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 314/631] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:50.843475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.844053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:50.844053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:50.844053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:50.866220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:50.867476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:50.959605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 318/632] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:50.960633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:50.992151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd1931-37ae-4916-a888-98255cdf18c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.011340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.045946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.046579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.046853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.047195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.067604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.069366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:51.086199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:51.087604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 315/633] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:51.088984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.106424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.138699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f511a5c-78cb-45f4-a7ab-b97d4a5aa778 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.185926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.186552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.186823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.187301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.198744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.200562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.216401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e14197-a9f0-4d62-86ed-dae7f6f8529d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:51.217030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:51.218144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:51.219232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:51.219232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:51.219232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:51.222804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 319/634] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:51.223701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.265359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.268829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.269586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.270218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.281655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:51.287841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:51.289038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:51.289959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:51.293946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 316/635] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:51.294551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.329917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.362847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87001af-5bea-4af4-8152-20f968a14b0a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.380265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.411175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.411777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.412052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.412322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.413144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:51.413435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.415022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.426890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3c7f0-9332-4cc6-81e4-55a1a4fec920 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.439478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 320/636] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:51.440378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.471644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.473207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.473643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.474340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.475580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:51.476274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.477827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.498260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 317/637] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:51.499494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.561490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb24d3e-d85d-4664-bca6-8c2339cf76ce None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.599251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.599852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.600124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.600452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.601352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:51.601470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.602782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.604523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.618530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 321/638] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:51.619461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.622198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da992f7-d69c-4544-8198-20491dad8ff0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.651077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.660159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.660761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.661036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.661303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.733887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.735412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aff844-b110-4b4d-90bd-1774c3c9738f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.737836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd1d0-c5b8-4c61-a27a-8d6c34b530fc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.775950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.776925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.777385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.777704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.786898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.788338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:51.799579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:51.800499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 322/639] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:51.801416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.850948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 318/640] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:51.852439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf26d0-a5f9-4e0e-a3e4-ee340af71708 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.852793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf26d0-a5f9-4e0e-a3e4-ee340af71708 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.853130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf26d0-a5f9-4e0e-a3e4-ee340af71708 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.858895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 319/641] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:51.858895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf26d0-a5f9-4e0e-a3e4-ee340af71708 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:51.871321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.890014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:51.922023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16990aea-697b-4744-85f5-1303e997be6a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.961680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:51.962326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:51.962623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:51.963093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:51.978335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf26d0-a5f9-4e0e-a3e4-ee340af71708 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:51.986817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:51.988411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:52.003968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:52.005051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:52.006266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:52.006266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:52.006266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:52.007868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 323/642] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:52.008893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.035648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.036518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.036944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.037362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.038241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:52.038714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.040310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf0bfd8-5860-4be3-a461-66fffc2dc57f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.062638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 320/643] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:43:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:52.064821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.065411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.065411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.085959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:52.087432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:52.135869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bd67f-9598-4277-8a1a-e9fa1177a389 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.147374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.184834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.185479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.185755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.186026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.199609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.201160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:52.218196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:52.219280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:52.225184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:52.225184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:52.225184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:52.225184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 324/644] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:52.225184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.288585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 321/645] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:52.289733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.318654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.345353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219bc8c-a614-4fe1-a3fd-7b8c815ce3cd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.391803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.392410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.392683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.393003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.393587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:52.393885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.395437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.411687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ed7fd-db84-4114-a73d-6a1ec8a56f14 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.412644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 325/646] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:52.413763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.445484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.450850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.451557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.451879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.452093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.535612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.537462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.541198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944e30-317e-410f-aecd-c3b7c13248e5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.556118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.581724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.582599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.583054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.583866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.584132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:52.584584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.585783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.602479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 326/647] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:52.602479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.655714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 322/648] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:52.655714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.696617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.718538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d71222-8368-40c1-ab76-d878b50aab03 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.755659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.756298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.756542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.756818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.759997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a9d3-76c0-4e9f-aea6-922ed2bc3ab2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.768308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:52.773762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:52.774927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 327/649] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:52.776037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.789346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.809763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.810640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.811092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.811514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.819187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.820621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:52.831789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:52.832935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:52.834030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 323/650] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:52.838076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.896944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948155ca-8d88-45af-9d1a-c5907ae1c7f4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.935049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.935687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.935968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.936240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:52.943546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:52.945033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:52.947718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:52.953302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b8fd9-11f9-4f84-8c59-ad213a096a85 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:52.957460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:52.958516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:52.959624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 328/651] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:52.960198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:52.990462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:52.991371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:52.991800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:52.992176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.003501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.004942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:53.019674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:53.020779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:53.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:53.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:53.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:53.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 324/652] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:53.021738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.050409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.073813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c466cb3-be95-48d9-9dba-180a99cc02bc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.108893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.109497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.109777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.110077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.110737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:53.111067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.112433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73842dac-9fe2-441c-bdab-9943360efc24 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.120392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 329/653] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:53.121586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa009-d9d5-45e8-b54f-9ed73bb5b707 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.121874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa009-d9d5-45e8-b54f-9ed73bb5b707 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.122148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa009-d9d5-45e8-b54f-9ed73bb5b707 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.127244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 330/654] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:53.127388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba90e-932e-4791-82fb-066755938a7d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.127512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fa009-d9d5-45e8-b54f-9ed73bb5b707 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.162817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.163816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.164275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.164713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.165953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:53.166444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.168908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.189397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 325/655] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:53.190220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.197407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.240655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fa009-d9d5-45e8-b54f-9ed73bb5b707 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.274797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.275418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.275694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.275961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.276641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:53.276938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.278347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.280948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.291644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 331/656] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:53.292738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.297115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6133603-0410-42fe-bd36-0c12f4ad40ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.337018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.337701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.337972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.338236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.338876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:53.339203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.340571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b187f9-f1a4-4997-a95b-2d92b4764f32 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.352928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 326/657] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:53.354201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.354589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.354729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.373598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:53.374790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:53.396506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fabd5-6c0c-4284-a42b-89b272513e74 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.419954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.432527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.433150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.433443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.433696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.445312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.446761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.448596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 327/658] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:53.449455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:53.465242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:53.466323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:53.467245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:53.467245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:53.467245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:53.472376 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 332/659] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:53.472376 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.556558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.574682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c844aa-18d6-4146-8fe9-fc18ed8552d9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.581497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645edcf-226c-4207-963e-4bd8d9229954 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.609942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.610545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.610820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.611390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.611859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=b46658fdbcde40498e87d37d79609c83) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:53.612163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.613597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.615826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.616458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.616853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.617299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.619852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 333/660] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/role_assignments?user.id=b46658fdbcde40498e87d37d79609c83&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:53.620661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.673235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.718732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.721651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.749605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0e44d-d2ae-469c-b6be-4167d1aa2af4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.795076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.796023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.796023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.796278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:53.805636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:53.808331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:53.824170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:53.825102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:53.825980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 334/661] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:53.826453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.832150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.874227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 328/662] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:53.875059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:53.944636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:53.948897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f413-7ad1-4d37-93c2-1b3e9188ce68 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.990488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa0253-f344-41e9-995f-ddcc8258356c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:53.991221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:53.991861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:53.992141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:53.992414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.010651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:54.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:54.016817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:54.017706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 335/663] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:54.018386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.040249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.040863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.041138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.041406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.042050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.042347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.043755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.057346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 329/664] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.058499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.106224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.160520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca97b2-13d8-464d-a77e-a7d12b076f3f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.195982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0724faa-f295-49af-9af6-1c3e5ea8df79 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.196361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.196963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.197241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.197517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.198213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.198521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.200010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.212725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 336/665] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.213639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.237637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.246523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.247517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.247945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.248364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.259935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.261616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:54.277330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:54.278372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:54.279753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:54.279753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:54.279753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:54.279753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 330/666] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:54.279987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.329809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e1695-6436-4b88-b3e0-4106f9f8c1a1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.368024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.368630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.368911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.369188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.369855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.370159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.371572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.380277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 337/667] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.381098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.385199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.404163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c76080-f9fb-4205-a05f-6696a6f179a5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.447739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.448623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.449052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.449547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.459277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.460935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.470604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:54.474601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:54.475765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 17:43:54.476946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:54.479161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 331/668] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:54.480010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.497957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26027-c479-4a8b-aba4-d0f2c973c4e3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.554873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.555601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.555880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.556154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.556882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.557186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.558610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.565050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 338/669] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.565868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.616834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae35de4-9b18-470b-958a-6c57f9cf318c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.668292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.668916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.669192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.669460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.670115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.670413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.671841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.693715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 332/670] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.693715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.705497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.715494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff49b1-18cb-4c04-ac46-e9e406ebc2c4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.763268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.764252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.764735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.765290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.778103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.779612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.784223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:54.802529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:54.805744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:54.806631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:54.806631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:54.806631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:54.808344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 339/671] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:54.809301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.832182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f7e5e-24d9-46c9-a6cb-d34123320789 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.881723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.882600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.882882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.883186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.883926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:54.884276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.885659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:54.914164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 333/672] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:54.915295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:54.944389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae84c-ee8f-4f85-86e0-67c0db6f0456 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:54.987809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:54.988461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:54.988854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:54.989196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:54.997113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:54.998811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.004512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:55.014544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:55.015603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:55.017224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 340/673] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:55.022578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.052700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153e7de-2e47-4968-8886-77788c9ac031 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.099871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.104062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.104995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.106362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.106871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.108101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:55.108927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.111381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.130274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 334/674] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:55.130274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.161176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348174f-4daa-41d7-9931-c9004e7fb4c8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.199535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.200473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.200932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.201391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.249585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04b7f-412c-479b-9dc2-3e9cde0bc28b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.278273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.288480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.288664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.289247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.289522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.289789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.289978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.290478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:55.290777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.292353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.307231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 335/675] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:55.308127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.349943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.399838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 341/676] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:55.400950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.425983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12697ed8-89ff-49b7-aab5-b5e759d4e5f6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.464092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.464822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.465194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.465466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.475924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.478604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:55.495734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:55.496740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 336/677] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:55.498554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.513254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac304af-8a18-4116-9866-2d9d0cc33f0a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.560659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.561197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.561832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.562095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.562886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.564416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3098494139cc41359d5c34b2a8428c6e) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:55.565137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.567431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.577260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 342/678] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/role_assignments?user.id=3098494139cc41359d5c34b2a8428c6e&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:55.578211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.613587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.624988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000b641-c6fc-4a6b-b6a0-5f458cf346ca None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.666987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.667764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.668144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.668512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.686118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.687613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.707358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040e01d-b055-46b1-856f-a4ac97ac02ae None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:55.708125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:55.710041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:55.711221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:55.711221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:55.711221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:55.711221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 337/679] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:55.724101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.747732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.748347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.748643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.748889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.749627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:55.750020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.752048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.772427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 343/680] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:55.780104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.848488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.854866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741885-806b-44be-87d5-b91151b46ecc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.907834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.908456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.908736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.909004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:55.909660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:55.909957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:55.911963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:55.916041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:55.931750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 338/681] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:55.938959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:55.947451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1b6e7-ced5-4e97-8785-766b5c4f97b6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:55.987919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:55.988871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:55.989296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:55.989817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.063057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32a079-41e9-4f3c-9901-1a6fbc8e0b09 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.065937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.067638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.102509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.103266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.103551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.103827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.112020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.113584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-18406fd7-0f0c-4675-b182-8ed56a3a705e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:56.126578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:56.127560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:56.128446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 339/682] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:56.131357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c958-7d27-4151-b020-565528527e51 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.131357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c958-7d27-4151-b020-565528527e51 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.131357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c958-7d27-4151-b020-565528527e51 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.131357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 340/683] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:56.136911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.137253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.137518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.159243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:56.160218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.161236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:56.177640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.192118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 344/684] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:43:56.193037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.242008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 341/685] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:56.247483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.304103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b7bb1-8eb6-4b96-8a6f-d1385587d9ee None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.343935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.345137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.345418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.345828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.346531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:56.346834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.348556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.368399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 345/686] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:56.369325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.374298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bb766-3ca5-47f3-b500-d309ec4d7b2c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.412835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.413870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.414386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.415051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.415753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.429160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.432297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.447486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:56.454316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:56.455341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:56.456240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:56.456240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:56.456240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:56.458451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 342/687] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:56.459520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.493210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f820-905b-4175-82ba-5aa6c7b1042f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.536373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.537291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.537579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.537774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.551781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.553389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:56.573785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:56.574852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:56.575767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:56.575767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:56.575767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:56.575767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 346/688] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:56.576428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.594877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bbb01-384d-4c53-8e4c-78e5f24d2728 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.633464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.634590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.635201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.635814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.636749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=768dabebad264236b8f43dbd1c85a964) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:56.637276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.639050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.646161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 343/689] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/role_assignments?user.id=768dabebad264236b8f43dbd1c85a964&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:56.647269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.728455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.731572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:56.745847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4066ca-bc4f-4307-aad7-0b109d5336f7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.767602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ed654-e213-4dc0-bee2-9808e62d07f8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:56.818634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.819288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.819567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.819953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.820614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:56.820915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.828562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:56.831530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:56.831530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:56.831530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:56.831530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:56.831530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:56.832745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.834351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:56.854774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 347/690] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:56.857369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:56.862680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 344/691] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:56.868762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.012338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9993b7-5a73-4535-8264-88e9ac91870d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.013428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb80980-238d-4629-9733-9e7eb2db73c4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.046193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:57.059271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.059879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.060160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.060430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.061091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.062756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.075043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.075043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.075043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.075043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.077105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.078265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0477ac9b-6201-4b08-afa6-a093bbe1f23b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:43:57.091772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:57.092853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:57.093946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 348/692] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:57.095103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ab2-ca87-4c5e-8e73-e062b06a760c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.099569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ab2-ca87-4c5e-8e73-e062b06a760c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.100063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd5ab2-ca87-4c5e-8e73-e062b06a760c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.100323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 345/693] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:43:56 2022] POST /identity/v3/users => generated 254 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:43:57.102297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 349/694] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:57.108587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.110566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.110961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.111859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.159837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:57.161349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:57.173273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:57.269393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303b412-b419-4b23-8ed3-53ea75204c5b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.296557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 350/695] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:57.311801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.312400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.312671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.313485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.314229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:57.314526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.316154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.324866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.332954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 346/696] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:57.340039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.392642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:57.465978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ec7fc-fd71-4eec-b7dc-dffbcd0307be None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.470964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714371b-5418-49c8-9c1a-0346cc651290 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.509132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.509347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.509622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.509884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.524155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:57.526762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.527949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.528119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:43:57.528594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.529113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.530042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=eae392d38ade4665b2385bae93b3e688) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:57.530546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.533515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:43:57.534032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:57.534937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:57.535811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 351/697] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:57.541319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.544052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 347/698] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/role_assignments?user.id=eae392d38ade4665b2385bae93b3e688&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:57.674535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ec997-0c5e-454b-a72e-d4d9a8ef94fc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.722930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.723613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.723983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.724314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.724991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:57.725287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.726742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.741337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 352/699] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:57.747419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:57.865470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cab22-a34a-45ea-92ce-43ef74bd37ee None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:57.868701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:57.905055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:57.906070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:57.906557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:57.907617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:57.919415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:57.921223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7e1acc12-5ccd-4d9f-becc-9ca7afef8f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:57.937442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:57.938456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:57.939367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:57.939367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:57.939367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:57.941442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 348/700] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:57.947192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.026011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:58.040746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2bc57-1ca6-49f7-bde7-52892a82df23 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.041076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2bc57-1ca6-49f7-bde7-52892a82df23 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.041311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2bc57-1ca6-49f7-bde7-52892a82df23 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.042918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 349/701] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:58.048621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.048907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.049181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.070952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:58.072106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f956197-7dc0-41f5-a0bc-9c01d9c6b803 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.072303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:58.116864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.118382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.118661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.118931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.119605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:58.119914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:58.121298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:58.137836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 353/702] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:58.144301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.163883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 350/703] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 17:43:58.184269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.279472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddbf7-636e-4f01-a8c9-70f667bf059e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.305573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:58.319487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.320684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.321304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.321920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.338344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:58.344000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:43:58.365954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:58.367115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:58.368053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 354/704] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:58.374485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.389964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:58.415562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16a2a6-6ac0-404f-a10f-8f7c4bf675ae None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.475456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.476514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.477000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.477442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.478489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:58.481292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:58.501497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:58.502574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:43:58.503428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:58.505603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 351/705] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:58.510216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce939ab-f031-418c-b750-c5ea29b87cee None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.525591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.610777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.611424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.611706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.611979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.614404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:58.614718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:58.616213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:58.645157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 355/706] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:58.650389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.720475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be215648-98a0-4450-9a10-bb35a5bc3f00 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.754159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:58.809094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.809717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.809998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.810271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:58.810918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:58.811269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:58.812648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:58.828803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 352/707] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:58.834188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:58.863966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:58.879194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023ad4-20c2-4ba3-acca-7be99ab1b18a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:58.950559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:58.951195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:58.951475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:58.951746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.049043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a69fe-5a4d-454b-9a2a-70d1eb4495a8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:59.103222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.104725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39e9ed-18f3-456e-8214-912bcdc7955a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.124240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:59.124777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.125055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.125330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.126007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:59.126311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.127716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.142755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 353/708] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:59.203054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:59.269513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 356/709] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:43:59.280144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:59.322323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f62ab-1ced-4841-937d-6501cf4032df None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.322694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f62ab-1ced-4841-937d-6501cf4032df None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.322879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f62ab-1ced-4841-937d-6501cf4032df None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.325060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 357/710] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:43:59.330871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.331433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.331716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.356169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:43:59.357386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:43:59.447474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:59.447996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 358/711] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:43:59.480537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e100ae6-1940-45c3-9fda-e9ea61452cb0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:59.490107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:59.568855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:59.569880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.570367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.570861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.593883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.599502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:43:59.632454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:59.633641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:43:59.634589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:43:59.634589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:43:59.634589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:43:59.643719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 354/712] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:43:59.644375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:59.651413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:59.685411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925b292-3bfe-4c79-9f60-b3aefa0f4312 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:59.725675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:59.726795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.727423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.728015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] PATH_INFO: `/v3/services/network` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.728994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.730599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:43:59.744996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:43:59.746057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:43:59.747041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:43:59.750372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 359/713] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:59 2022] GET /identity/v3/services/network => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:43:59.756443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:59.838753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a0c00-ef78-40ee-8e9d-87bd7f72b137 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:59.876229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:59.885120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:59.886461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.887096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.887723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.888679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:59.889098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.891810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69536f12-20f4-4b0e-b406-9019c664cb58 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.909061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 355/714] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:43:59.910316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122406a8-7b57-4240-993e-92a5e4ad78f9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.910604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122406a8-7b57-4240-993e-92a5e4ad78f9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.910880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122406a8-7b57-4240-993e-92a5e4ad78f9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.916320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 356/715] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:43:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:43:59.918902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122406a8-7b57-4240-993e-92a5e4ad78f9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:43:59.946687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e678909f-9edc-4c2d-b525-47d669766bde None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:43:59.982910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:43:59.983470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.983750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.984025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:43:59.984660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] RBAC: Adding query filter params (name=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:43:59.985503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:43:59.986903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fca6a-c77c-4b97-b30e-ff1606200d97 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:43:59.988269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:43:59.993232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 360/716] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:43:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:43:59.994504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:43:59.994784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:43:59.995101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.014420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:00.015793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:00.045588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122406a8-7b57-4240-993e-92a5e4ad78f9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.097520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.098184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.098467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.098784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.099293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 361/717] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:44:00.100217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.107123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:00.108574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:00.121975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:00.124193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:00.126439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 357/718] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:43:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:00.132062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.206651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:00.231034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79f20-0677-4fcc-a20b-083077c2969d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.262938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:00.276029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.277036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.277469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.277890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.278672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] RBAC: Adding query filter params (type=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:00.279200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:00.279418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d93c1b-726c-4054-92a3-a7a0b906881b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.281169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:00.288776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 362/719] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:00.289925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.328244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.329637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.330075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.330514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.331446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:00.331912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:00.333847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:00.357593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 358/720] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:00.364773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.470793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05529ff-375c-478d-b3bb-44cc358ec513 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.514843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.515891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.516937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.517409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.534073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ca46e5a2-bffd-46ab-8410-dc5733284aca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:00.539173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:00.540114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:00.540953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 363/721] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:00.542369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b9421-d69b-4c92-af22-809c343f7e15 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.542369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b9421-d69b-4c92-af22-809c343f7e15 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.542369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b9421-d69b-4c92-af22-809c343f7e15 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.543808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 364/722] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:00.546891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b9421-d69b-4c92-af22-809c343f7e15 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.557654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4478-4a49-49a2-8a09-350295202c41 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.573949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:00.603982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.604691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.605063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.605406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.606624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:00.607016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:00.608621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da13cb9-d531-47d2-81f5-f64a9f136294 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:00.614839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:00.618642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 359/723] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:00.625638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.626504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.627206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.661955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:00.663623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:00.684619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b9421-d69b-4c92-af22-809c343f7e15 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.745903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 360/724] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:44:00.776701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.777251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.777538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.777814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.778510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:00.778818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:00.780274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:00.786435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.795026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 365/725] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:00.802120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:00.932984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e778c2-b7d9-4fc0-81e3-00e0304aa201 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.936926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:00.956965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a6b21-2d02-4b67-92e6-75a7ee7b6638 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:00.985282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:00.986506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:00.987276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:00.988065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:00.996614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:01.013126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:01.025373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:01.026556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:01.027639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:01.029890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 361/726] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:01.035923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.051923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.052566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.052841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.053109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.080454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.081973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:01.122495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:01.123741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:01.124736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:01.124736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:01.124736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:01.124736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 366/727] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:01.125827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.213911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937848fa-86cd-4217-b57e-f17a4567d8f1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.277513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103c350-7f0f-44d6-bb65-2bc7e18415f8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.306459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.307188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.307568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.307940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.308710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:01.309112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.310591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.326998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 362/728] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:01.328206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.333344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.334025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.334363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.334539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.335235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:01.335537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.336927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.351842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 367/729] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:01.356602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.387599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:01.425005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:01.527610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb5d1-cf8f-40ae-a0e5-e21bc141bd7f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.565664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.566606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.566885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.567265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.576211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.577288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fb6b9-199b-4ef6-bea8-272d6ee13d5d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.577665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:01.591451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:01.593575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:01.595977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 368/730] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:01.604295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.629036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.630068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.630566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.631130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.643985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.645849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:01.662932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:01.663987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:01.664876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:01.664876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:01.664876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:01.667989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 363/731] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:01.674686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.741044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:01.760556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67354bcc-c4f5-4d80-9188-53736a13b4f1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.779629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:01.806760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.807393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.807670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.807941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.808487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250205de-ff14-49cb-b2a0-ff18bb9a8c47 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.808675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:01.808979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.810692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be5706-11f3-4fdf-a67b-1b0de8b87034 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.838889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 369/732] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:01.840741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3a-c113-4e86-b3e5-c4662063551e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.841026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3a-c113-4e86-b3e5-c4662063551e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.841304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b1c3a-c113-4e86-b3e5-c4662063551e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.843352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 370/733] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:01.844664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1c3a-c113-4e86-b3e5-c4662063551e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:01.858213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:01.858902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.859587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.860483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.861567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:01.861878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:01.863815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cd3f3-d171-4cbc-aaf3-afedba6109a3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:01.879532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 364/734] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:01.880810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:01.881093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:01.881364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:01.902644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:01.904227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:01.977424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b1c3a-c113-4e86-b3e5-c4662063551e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:01.991645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 365/735] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:44:01.994508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.029457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.030067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.030412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.030677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.048138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.100308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.123738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.125322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.147593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad651-bb1f-4247-a0e3-cde3a6ae143e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.197441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.198954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.199426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.200128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.213782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.216209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:02.234285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:02.236040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:02.237022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:02.237558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 371/736] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 17:44:02.238733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.239648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 366/737] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:02.243950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.364037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cb80-eb0f-4f58-a574-70d959a234a0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.394941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.405061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.406126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.407177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.407744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.410826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.420807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.423052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:02.441235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:02.442367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:02.443547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:02.443547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:02.443547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:02.443547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 367/738] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:02.444965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.479181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa0d06-3435-4441-ae10-88cc84deb2b7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.526642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.527606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.528028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.528435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.529221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.531010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.554420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 372/739] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:02 2022] POST /identity/v3/services => generated 251 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:02.556218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.570562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54ffe3-178e-428c-b686-10b89b627643 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.624716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.625842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.626412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.627003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.628510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:02.629158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.631104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add88fd-42c4-463e-8f5b-6ad729423650 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.657097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 368/740] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:44:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:02.657097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9c0a4-b5cd-4b52-8acb-18a47df2620b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.657097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9c0a4-b5cd-4b52-8acb-18a47df2620b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.659551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9c0a4-b5cd-4b52-8acb-18a47df2620b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.661382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 369/741] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:02.663825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9c0a4-b5cd-4b52-8acb-18a47df2620b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.705698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.715079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2b0c-4de5-4b7b-b810-8093d36c3d70 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.718269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:02.761218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.761928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.762207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.762475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.763197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:02.763507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.764865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10e323-ad43-4e62-a3d0-76f6c8975eee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.782602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 373/742] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:02.783919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b479-456b-4066-976d-603a45910da7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.784163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b479-456b-4066-976d-603a45910da7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.784436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b479-456b-4066-976d-603a45910da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.797042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9c0a4-b5cd-4b52-8acb-18a47df2620b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:02.805879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-46d8b479-456b-4066-976d-603a45910da7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:02.807128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-46d8b479-456b-4066-976d-603a45910da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:02.839063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:02.840221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:02.840642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:02.841049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:02.885014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 374/743] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 17:44:02.892234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b479-456b-4066-976d-603a45910da7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:02.928153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:02.929627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:02.995499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.029077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.039385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b479-456b-4066-976d-603a45910da7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.079028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 370/744] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 17:44:03.079028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.092255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.093235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.093828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.094995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.102319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.104727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:03.128207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:03.129163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:03.130037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:03.130930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 375/745] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:03.132183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.186663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ac4b9-62ac-4bf4-8b1c-a96772e12fda None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.230219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.231445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.231908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.232320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.233113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.234542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05fa9a-7108-44d1-b4be-356a4c14539f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.238069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.253488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813892aa-7184-4342-b0cd-4d80b3b107fb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.256951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 371/746] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:03 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:03.257254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bc46-df90-4061-884a-5ff0d518f4d6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.257543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bc46-df90-4061-884a-5ff0d518f4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.257814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4bc46-df90-4061-884a-5ff0d518f4d6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.263479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 372/747] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:03.264456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4bc46-df90-4061-884a-5ff0d518f4d6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.319950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.321019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.321551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.322057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.335889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.337855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e6d5675f-45b2-4ef7-87cf-fce8e6883485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:03.356587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:03.357594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:03.358493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:03.358493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:03.358493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:03.365543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 376/748] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:03.368022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.368688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.369099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.377799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.392689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:03.393983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:03.395397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4bc46-df90-4061-884a-5ff0d518f4d6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.434102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.434829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.435143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.436135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.436248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:03.436511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.438534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.452100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 373/749] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:03.454655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.485957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 377/750] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:03.486814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.529082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.595867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0b005-6850-43ee-a456-8e5ef55aec24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.617942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e052313-d9b2-4bb1-b3e7-e5f0c80d9158 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.645229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.646010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.646426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.646701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.647631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:03.647936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.650061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.654698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.666684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 374/751] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:03.667611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.668872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490be5-b799-424e-babe-03989d56202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.670342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490be5-b799-424e-babe-03989d56202a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.670792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490be5-b799-424e-babe-03989d56202a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.671191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59490be5-b799-424e-babe-03989d56202a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.671969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59490be5-b799-424e-babe-03989d56202a None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3a73ecd2399f434ba5999aa5be2f7fd9) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:03.673042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59490be5-b799-424e-babe-03989d56202a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.675623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59490be5-b799-424e-babe-03989d56202a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.682959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 378/752] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/role_assignments?user.id=3a73ecd2399f434ba5999aa5be2f7fd9&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:03.684004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490be5-b799-424e-babe-03989d56202a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.835147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db63d-599b-4150-b39e-495b596b7982 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.863368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:03.888753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59490be5-b799-424e-babe-03989d56202a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:03.906392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.907992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.908472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.908939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.920994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:03.927636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:03.928473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:03.931078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 375/753] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:03.937694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:03.944077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:03.944713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:03.944987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:03.945256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:03.960082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:03.961562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:03.985334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:03.986278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:03.987180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:03.995454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 379/754] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:03.996354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:04.051128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91567bd-1717-4d12-ad4a-07b296257d4e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.112047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.113095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.114414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.115128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.117264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:04.117802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.119488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.134089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 376/755] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:04.136794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:04.141673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:04.223394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:04.251212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08279e-a298-4205-93e5-184412118db4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.258684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7f367-f7e7-4ec8-a771-0af565f8456d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.305374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.306325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.306860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.307374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.308426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:04.308948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.310719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.324962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 380/756] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:04.332010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:04.340592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.341237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.341517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.341786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.365077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.366455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:04.397567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:04.399220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:04.400321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:04.400321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:04.400321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:04.400321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 377/757] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:04.404276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:04.461032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d53edd-f90d-45eb-b4d1-374955aed026 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.488599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:04.531103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:04.547615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730d67-933e-4947-b15a-43cb854a6f90 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.578923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.579797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.580199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.580513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.581335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=eae392d38ade4665b2385bae93b3e688) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:04.581659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.585919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.598096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 381/758] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:04 2022] GET /identity/v3/role_assignments?user.id=eae392d38ade4665b2385bae93b3e688&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:04.612551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.613467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.613892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.614339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.615236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:04.615718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.631685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.647535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 378/759] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:04.653655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:04.805379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845b862-f6ba-4aae-8ebf-22e4b7af81e5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:04.843969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:04.843969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:04.844247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:04.844516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:04.852741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:04.854192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-a335987e-b9f5-49d1-9021-70663aa16a90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:04.871643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:04.872612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:04.873562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 382/760] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:44:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:04.880643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.052694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcc1e0-38c6-4db2-bd0b-3965e0b08f9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.053066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae54fe-9f29-455a-9fe7-10ecb8a6da1f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.053253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae54fe-9f29-455a-9fe7-10ecb8a6da1f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.053481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae54fe-9f29-455a-9fe7-10ecb8a6da1f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.054924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 383/761] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:05.061871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.061871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.061871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.096631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:05.098892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:05.108052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.108951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.109450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.110527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.111469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:05.111950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.113547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.144378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 379/762] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:44:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:05.147898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.163348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:05.193596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:05.257035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 384/763] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:05.292439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.341286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e58c2e-750d-4546-baf1-73fe3ff91237 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.385192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.386136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.386596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.387092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.400035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.402522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:05.418856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:05.419912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:05.420845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:05.420845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:05.420845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:05.423547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 380/764] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:05.429327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.488210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e968c-4c36-4a1a-a938-0ec3a2f71e0c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.494380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:05.541861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4caede-541a-4f70-aab1-cd8d6159cf49 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.567874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.568515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.568797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.569070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.569757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.571492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.584503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.585511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.585938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.586350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.587177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:05.587815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:05.589294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:05.590249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:05.591103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 385/765] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:05.593418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.597539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.608630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 381/766] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:05.617754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.671542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:05.730132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda44d-f6fd-44bb-9ccd-4748958ed273 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.755358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:05.769912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.771443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.772021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.772645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.782966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.784801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:05.797998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:05.798939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:05.799851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:05.802786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 382/767] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:05.808762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.834847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab1fe-df2f-4e17-a1b5-00348234b3c6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.929149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ae7a4-1799-44d8-b5d6-99235667b4b6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:05.936672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.937299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.937580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.937891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.938531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:05.938834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.940508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf5d3c-04c7-429c-9e94-0024314df83c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.945826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 386/768] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:05.947160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999133-77bf-4364-bfd8-197ef11e5273 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.947367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999133-77bf-4364-bfd8-197ef11e5273 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.947637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd999133-77bf-4364-bfd8-197ef11e5273 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.949100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 387/769] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:05.951763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd999133-77bf-4364-bfd8-197ef11e5273 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:05.973127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:05.973744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.974025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.974294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:05.975016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:05.975327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:05.976772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d595bb-2739-45fe-bca9-3b844700060f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:05.991158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 383/770] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:05.992557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:05.992847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:05.993167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.013177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:06.014404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:06.047929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.097122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 384/771] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:06.097317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.097427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.100552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd999133-77bf-4364-bfd8-197ef11e5273 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.152262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.153286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.153768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.154323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.155187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:06.155687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.157381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.163820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 388/772] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:44:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:06.165016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.238229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688672d2-f665-4e94-ae13-1f4713db9912 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.274682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.275389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.275668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.275945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.277815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=a136641c41464f1f969fe6c16dfe3501, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.279276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.280914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02012dae-8d45-40c3-a6fa-b36661c58d16 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.293364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 385/773] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:06 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:06.293961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.329065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.333910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.335940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.336913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.337610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] PATH_INFO: `/v3/services/network` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.339068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.342573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-58f4aa83-4b59-4cef-8ca5-4665215de28a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:06.364064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:06.365747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:06.367180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:06.371250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 389/774] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:06.373911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19282f-baa1-4ceb-aefd-61b2f84e5196 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.375778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19282f-baa1-4ceb-aefd-61b2f84e5196 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.376077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19282f-baa1-4ceb-aefd-61b2f84e5196 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.377750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 390/775] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:06.378262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19282f-baa1-4ceb-aefd-61b2f84e5196 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.430778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.433094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2006931-daa0-4c69-95ab-37d860e06c93 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.480541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.481460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.481732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.481995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.483186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] RBAC: Adding query filter params (interface=public, service_id=42f869741eeb4f74b6819dddab8f2c38, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:06.483502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.485693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efe199-b952-4010-b522-bccd1a702642 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.498463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 386/776] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/endpoints?service_id=42f869741eeb4f74b6819dddab8f2c38&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:06.499776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.500063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.500909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.523349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:06.525163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:06.528776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19282f-baa1-4ceb-aefd-61b2f84e5196 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.571502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.572153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.572264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.572514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.573149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] RBAC: Adding query filter params (name=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:06.573455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.574804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.580409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 391/777] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:06.581230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.623427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 387/778] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:06.626677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.676729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.717981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:06.731106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281de8e7-669f-4f4e-a02b-0228f30f4287 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.774340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.774862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.775173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.775539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.776051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.777419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.783615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 392/779] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:06.784798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.791365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee55982-069a-4e11-9108-5d11c07f67a8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:06.838095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:06.839007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:06.839439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:06.840466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:06.841311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:06.841799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:06.843585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:06.851873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 388/780] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:06.858361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:06.974841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:07.001573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7442ee-54ca-49fb-a1d5-ebcd579870f0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.024268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f1a79-74d0-40d9-b85c-03ccb64fba18 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.048720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.049675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.050157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.050646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.051763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] RBAC: Adding query filter params (interface=public, service_id=7132287352924799b3b75b6a3d72d488, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:07.052284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:07.054168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a3936-9fcd-42a2-99cb-c98398cf67ae None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:07.061008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 389/781] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/endpoints?service_id=7132287352924799b3b75b6a3d72d488&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:07.064335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1761ae-eb2e-4eea-aa8d-71436005284e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.064805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1761ae-eb2e-4eea-aa8d-71436005284e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.065243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1761ae-eb2e-4eea-aa8d-71436005284e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.067205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 390/782] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:07.068150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1761ae-eb2e-4eea-aa8d-71436005284e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.097829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.098473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.098730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.099015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.111380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8b285b3a-80d6-455f-80d7-f2d32c8083c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:07.116375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:07.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:07.118464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 393/783] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:44:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:07.119175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.123966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.124252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.163502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:07.164699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:07.176812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:07.251533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 394/784] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:07.253692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.317147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1761ae-eb2e-4eea-aa8d-71436005284e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.367354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.368450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.368927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.369544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.370402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:07.372355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:07.378918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 391/785] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/services => generated 772 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:07.380086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.427279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:07.557629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95781ab9-9767-4a9b-b233-0137980d6b90 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.608629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.610074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.610547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.611852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.612728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:07.613224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:07.615284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:07.627663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 395/786] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:07.632998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.644961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:07.648462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcc343-2f51-463b-a0ab-a8ee03002294 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.700110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.701138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.701628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.702171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.714738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:07.720113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:07.721058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:07.721876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:07.724618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 392/787] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:07.735117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.852334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf55eb-ec64-4b7e-acb1-51215e0ddbe7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:07.879776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:07.899442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:07.900509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:07.901008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:07.901570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:07.914673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:07.916659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:07.934894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:07.936067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:07.936955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:07.936955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:07.936955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:07.940350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 396/788] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:07.946465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:07.949664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442c67a-a158-4621-8eb3-77f6f576c9e0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.025268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.026269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.026750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.027244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.028131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:08.028659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.030228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.047820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 393/789] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:08.052005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.071123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:08.118961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9323f-67fe-47bc-be13-183ee9de92a9 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.162011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.163189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.163647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.164050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.165220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:08.165673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.167230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.190179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 397/790] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:08.196074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.235172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc3715-59a7-495d-ab58-50c8a5480c27 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.287019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.288037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.288491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.288902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.292354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:08.309157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.311043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.322963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b2fe7-5c8e-4548-a172-eeaadcb739c6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:08.344499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:08.345801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:08.346889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:08.346889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:08.346889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:08.346889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 394/791] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:08.349296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:08.355285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.387033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.387674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.387952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.388222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.401619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.403914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:08.431321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:08.432455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:08.433575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:08.434477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 398/792] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:08.440314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.551145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0846fd3-02a2-4952-898d-e9287ced4b3e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.586284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.587145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.587507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.587901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.588754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:08.589139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.590643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.602812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2210a0d-790b-4744-ac5b-95fbf684a902 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.611548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 399/793] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:08.624165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.644483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.645137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.645416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.645680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.646318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:08.646613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.647998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.666062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 395/794] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:08.667357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.695216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:08.723755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:08.783836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141887f8-14a1-4ed1-b5d5-b980dd627f71 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.790686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261d68d-a566-4fa4-afab-4f7a322e9c83 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:08.819832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.820330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.820601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.820864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.829123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.830414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.838438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:08.839416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.839923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.840438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-df6e4563-f2bf-4bd3-b341-7155a977bbfb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:08.843371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:08.844864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:08.845829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:08.849857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 396/795] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:08.851377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc46aa-3199-4be5-9d5b-e913a42476a7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.852064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc46aa-3199-4be5-9d5b-e913a42476a7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.852064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc46aa-3199-4be5-9d5b-e913a42476a7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.853497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 397/796] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:08.857990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc46aa-3199-4be5-9d5b-e913a42476a7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:08.862329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:08.864112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7c470717-8247-4d83-b890-eed01f458a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:08.885371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:08.886357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:08.887267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:08.887267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:08.887267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:08.889433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 400/797] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:08.890653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:08.891484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:08.891751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:08.914672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:08.916050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:08.994317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 401/798] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:08.997175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.000740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc46aa-3199-4be5-9d5b-e913a42476a7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.013267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.036943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.053559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.055539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.056009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.056449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.057380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:09.057850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.059700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.086705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 398/799] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:09.087876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.150863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f95b3-78d1-41d2-bb3c-d4455b24ac32 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.192758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.193495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.193836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.194168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.195009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:09.195258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.196824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.210209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 402/800] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:09.211307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.225250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81018c3-9af6-482a-883d-e3516df2380e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.274867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.276083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.276961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.277568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.293394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:09.299054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:09.300011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:09.300838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:09.303691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 399/801] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:09.304703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.308799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.352757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.360799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ce94-e32c-43dc-9818-c16a2668f998 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.404681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.405327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.405603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.405877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.417974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.419544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:09.440283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:09.441546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:09.442515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:09.442515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:09.442515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:09.442515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 403/802] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:09.442515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.444910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddeead-7a27-4f60-af4a-1845ef9dd402 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.481908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.482532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.482806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.483105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.490870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.492454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f3d64330-0d7e-4c32-8268-c02237d2cfc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:09.507184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:09.508637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:09.509540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 400/803] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:09.510029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004b51-13ae-4cc4-ba94-6ab80ac2d9e3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.510029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004b51-13ae-4cc4-ba94-6ab80ac2d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.510150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c004b51-13ae-4cc4-ba94-6ab80ac2d9e3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.515209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 401/804] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:09.515209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c004b51-13ae-4cc4-ba94-6ab80ac2d9e3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.563838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf33b3-26a7-45c7-a83f-39958003ca6a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.595568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.608131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.608760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.609043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.609312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.614553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:09.614553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.614553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.621649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.621802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 404/805] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:09.622497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.654697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c004b51-13ae-4cc4-ba94-6ab80ac2d9e3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.695831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.696701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.696827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.697073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.697858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:09.698048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.699429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae994a8-3032-4c43-b406-7c6b31d1e9ea None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.712005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 402/806] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:09.713390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474d050-5267-457a-9917-9472d7f16e63 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.713390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474d050-5267-457a-9917-9472d7f16e63 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.717244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474d050-5267-457a-9917-9472d7f16e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.734157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9474d050-5267-457a-9917-9472d7f16e63 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:09.735913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9474d050-5267-457a-9917-9472d7f16e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:09.756442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec8688-abf5-48e9-a52b-52ee77967488 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.794767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.795402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.795680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.795950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.796637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:09.797450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.798877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.813914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 405/807] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:09.814655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.820915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 403/808] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:09.822299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c534a-867d-48a1-ba43-2dfb9ebca765 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.822580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c534a-867d-48a1-ba43-2dfb9ebca765 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.823334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c534a-867d-48a1-ba43-2dfb9ebca765 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.828571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 404/809] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:09.828571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c534a-867d-48a1-ba43-2dfb9ebca765 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.868960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.911272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:09.917898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789285-06ed-4556-b68a-0f8b0e1f939d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.936383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c534a-867d-48a1-ba43-2dfb9ebca765 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:09.952336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.952973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.953276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.953556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.963819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.965267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.976522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:09.977143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:09.977419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:09.977688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:09.980216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:09.981499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:09.987991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:09.987991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:09.987991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:09.987991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 406/810] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:09.987991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:09.988279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:09.988279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-025860a6-f033-4e15-9983-cf52d24074f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:09.998127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:09.999083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:10.000500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 405/811] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:10.001084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.001084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.001345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.018682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:10.019902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:10.090202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a34674-5697-4b3b-a762-850d853e8f05 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.092549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.099048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 406/812] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:10.099895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.131638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.132300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.132577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.132849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3a73ecd2399f434ba5999aa5be2f7fd9/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.148867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=3a73ecd2399f434ba5999aa5be2f7fd9, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.150308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.163563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 407/813] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:09 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3a73ecd2399f434ba5999aa5be2f7fd9/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:10.164448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.177059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.223688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ab99b-d8d7-4ca8-85e4-77510800a4dd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.270582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.271365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.271647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.271914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.272508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.273903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:10.285407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:10.286618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:10.287896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 407/814] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:10.288685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.294128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955729-0989-4660-9424-9a08347d4af2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.334474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.335129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.335412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.335684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.336364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:10.336659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.338037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.355068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 408/815] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:10.356267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.375423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.406345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ba5b2-2635-474d-b2ca-a6fa2a6c297c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.435657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.442881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.443593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.443870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.444141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.444864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:10.445263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.447030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.466815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 408/816] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:10.466815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.482579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5eb4d-0e47-4ee3-9c42-cfffc9302b6c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.520491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.521211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.521554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.521892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.522544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.523996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:10.533718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:10.534957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 17:44:10.536025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:10.540697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 409/817] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:10.541421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.577534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7b153-5206-4bae-82d7-e4434d8a1fd3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.614359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.614959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.615295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.615570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.616198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:10.616499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.617867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.627534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 409/818] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:10.627534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.645506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.677402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a421d-3289-4952-8ba9-872e13dc3fa0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.695888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.716740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.717667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.718125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.719180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.737238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.739009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.744158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b5a10-1fa1-4dd4-b4d6-c28adcd7e6ca None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:10.753428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:10.754396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:10.755359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:10.755892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 410/819] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:10.756895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.788171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.789082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.789547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.789821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/eae392d38ade4665b2385bae93b3e688/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.791785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=eae392d38ade4665b2385bae93b3e688, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.793219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.808528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 410/820] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:10 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/eae392d38ade4665b2385bae93b3e688/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:10.809362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.893877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ccb7a-7ebd-41ce-ba91-a64cb42c35fd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.943223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.944295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.944779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.945319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.946222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] RBAC: Adding query filter params (name=network) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:10.946347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0542-a1b8-4bd0-b0d5-806b7ac846ef None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:10.946878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.948532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.955033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 411/821] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:10.956212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:10.959454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.963845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:10.987582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:10.988201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:10.988482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:10.988846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:10.989654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:10.989960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:10.991911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:10.998710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 411/822] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:10.999707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:11.097176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cf40-5a62-4041-a337-f322fa94ac23 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.113344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16ae12-e3d2-4bc3-a438-d0ea51965f8f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.152159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.153153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.153578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.153993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.154873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:11.155491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.157119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.157405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.157937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.158311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.158679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.159633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] RBAC: Adding query filter params (type=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:11.160040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.161776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.167312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 412/823] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:11.168516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:11.180083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 412/824] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:44:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:11.181259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:11.216809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:11.267396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:11.280200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838d852-8625-42d2-abbd-71150e34fa1e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.303144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ebd0d-68cb-461c-8743-1a40bdc69b0c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.319061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.319785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.320130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.320478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.321219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.322829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.349979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 413/825] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:11.351822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:11.356715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.357669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.358102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.358478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.359394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.361615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.386546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 413/826] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:11.483529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:11.491967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e65119-6b82-4704-bc87-55859c3f01fb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.542047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.542682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.542961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.543257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.624426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.625912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.738479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:11.748507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 414/827] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:11.754652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:11.867811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f682c727-f7e7-42ed-be4d-c2ccd7e63f85 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:11.884769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:11.908433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:11.909070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:11.909346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:11.909618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:11.921240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:11.922801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:11.937734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:11.938715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:11.939607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:11.939607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:11.939607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:11.939607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 414/828] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:11.945880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.053042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d0f23-0a7a-4f9e-b0c1-fdf2e112d083 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.091462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.092421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.092957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.093439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.094891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:12.095409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:12.096962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:12.109970 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 415/829] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:12.115683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.226771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ec1f8-ccb0-443a-bf90-6b1ec48d0601 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.264218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.265190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.266275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.266777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.274146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:12.276009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-bc922eb3-9dd0-4987-b062-fa1f83036a84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:12.292058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:12.293082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:12.293985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 415/830] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:12.301027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.310737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8d62-c2da-4707-9dd5-1273a862a590 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.311009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8d62-c2da-4707-9dd5-1273a862a590 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.311248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf8d62-c2da-4707-9dd5-1273a862a590 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.313228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 416/831] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:12.318780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.319092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.319366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.344398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:12.345914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:12.366831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:12.372211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:12.419910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349935e3-6163-4cb5-883a-ad40ac432d5a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.446404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 417/832] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:12.460691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.461500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.461919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.462341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.463257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:12.463721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:12.465413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:12.466224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.479560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 416/833] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:12.486556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.572874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d29d7-25e9-49f9-90a2-293279ff3247 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.593750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a6493-152a-474e-836b-a637c79039af None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.609779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:12.610120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.610869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.611286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.611647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.622758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:12.627884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:12.629103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:12.630013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 418/834] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:12.636336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.639124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.639933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.640412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.641444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.642361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:12.642832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:12.644583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:12.655282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:12.656962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 417/835] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:12.771324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cfb9c-8017-4366-8473-4f48f9b9eb14 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.813869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.814479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.814816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.815120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:12.815807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:12.816106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:12.817511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:12.828720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 419/836] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:12.834404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:12.947141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db59ee-32a3-42b1-9eba-0409bb955506 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:12.983453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:12.984495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:12.985037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:12.985611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.001666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.001666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7d92857a-49f9-49a2-9349-2f566a98aa3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:13.014629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:13.016205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:13.017099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:13.017099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:13.017099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:13.017099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 418/837] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:13.022238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.105548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.130859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde595ee-fab3-49bd-b40b-629d954cec86 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.165510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.166115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.166391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.166661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.167326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:13.167629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.169116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.178281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd1a59-d48d-4262-ace0-72914ccfe630 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.178567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd1a59-d48d-4262-ace0-72914ccfe630 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.178893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcd1a59-d48d-4262-ace0-72914ccfe630 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.180433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 419/838] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:13.185263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 420/839] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:13.185778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.186332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.186798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.191287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.207820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:13.208693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:13.233960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.284958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 420/840] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:13.308205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.339977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719108b-2b26-4e08-918e-efb8cfec4a79 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.388554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.389959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.390237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.390527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.398794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.406438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.408211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:13.424834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:13.425797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:13.426812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:13.429200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 421/841] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:13.435249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.445868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcd869-e114-42df-baf3-15b1df4ddd9e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.493037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.494162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.494687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.495251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.507742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:13.513264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:13.514205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:13.515070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:13.523088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 421/842] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:13.524221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.549018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.553325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d801ac-f9cc-429e-a7a8-9d8f16034eeb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.589180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.589736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.590022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.590293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.591210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:13.591413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.592911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.612943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 422/843] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:13.618127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.640992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8ed8-5de6-494b-a2c5-d2146396f52c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.678458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.681298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.682180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.682668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.683186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.684050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:13.684576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.686163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.700697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 422/844] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:13.701900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.727097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42159b-e811-4436-aac0-6f365aa5901a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.764207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.764824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.765141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.765483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.806700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.831817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa231f-f511-4825-b80d-7ef6f7849325 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:13.858722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.860736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33a8e2-0091-4fb5-b7fc-6bd8191a109a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.873617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:13.874520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.874793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.875088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.886876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:13.889141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0cff84b4-58d1-46a5-ba21-250241a496aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:13.908006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:13.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:13.910866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:13.910866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:13.910866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:13.911431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 423/845] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:13.913330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9eba-73a9-4a91-b94d-424d39fb6632 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.913824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9eba-73a9-4a91-b94d-424d39fb6632 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.914001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9eba-73a9-4a91-b94d-424d39fb6632 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:13.916518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 424/846] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:13.916518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9eba-73a9-4a91-b94d-424d39fb6632 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:13.930067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:13.996448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 423/847] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:44:13.997874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:13.998163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:13.998439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.026732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:14.027968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:14.044325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9eba-73a9-4a91-b94d-424d39fb6632 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.083107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.083813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.084088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.084357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.085044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:14.085343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.086802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.088392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.100791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 425/848] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:14.101609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.111782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 424/849] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:14.112817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.208059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.228117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b175292-797d-4ce1-8a3c-21de50f7aca0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.228317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152579d-04cc-434d-a631-8d82b59d5594 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.274253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.275265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.275708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.275905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.276142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.276838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.277257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.277675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.289969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.292293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.293995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.296282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:14.311276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:14.312716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 425/850] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:14.319386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:14.320061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:14.322185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:14.325701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:14.325701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:14.325701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:14.325701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 426/851] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:14.325701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.370731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.429365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46fb6b-2106-476c-89a7-3796d52ef2b6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.443260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9647c9-f5f4-43a3-a50c-599b38cfa4c7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.476570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.477176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.477520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.477857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.478652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:14.479147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.480849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.485730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.486517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.486781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.487118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.487775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.489210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:14.498916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:14.501030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:14.502928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 426/852] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:14.505058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.515103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 427/853] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:14.515103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.518152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.619276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ae2e5-019c-4b0a-8297-71b20c20fe67 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.621615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.626590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f689b2-e9d4-48b4-97d1-23f373f54e99 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.657402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.658010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.658292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.658566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.659231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:14.659536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.660898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.665416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 428/854] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:14.666060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.666180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.666997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.667404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.667793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.668722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:14.669020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.670682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.685013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 427/855] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:14.685943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.774342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.796355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5605f-45d2-4a34-88ac-d1c68a67ac41 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.810656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331d8ea-6eea-4bc9-97b6-1093ac760bc6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:14.843068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.844259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.844557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.844829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.848078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:14.849515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:14.849798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:14.850071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:14.850702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:14.851024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.852393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.858161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 428/856] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:14.859103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:14.885010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:14.933184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:14.934677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f2f80-6e44-4ac4-88f9-7f70acde7cc7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:14.981607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b286067-d37d-4588-becb-8010d377ffe8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.027289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.034027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.034664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.034943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.035249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.042618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.044238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ad74d8d5-272e-4d71-a594-2d5b92340791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:15.056272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:15.057202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:15.058093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 429/857] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:15.059144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f894-e394-4bfc-bbb8-8caee3af2ce1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.059377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f894-e394-4bfc-bbb8-8caee3af2ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.059646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f894-e394-4bfc-bbb8-8caee3af2ce1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.061014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 430/858] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:15.063902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f894-e394-4bfc-bbb8-8caee3af2ce1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.065874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 429/859] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:15.069139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.069688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.070164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.095391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:15.096631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:15.164331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.171478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 430/860] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:15.172171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.192388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69f894-e394-4bfc-bbb8-8caee3af2ce1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.229163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.229767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.230036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.230312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.231297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:15.231600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.233015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.245147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 431/861] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:15.245955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.293301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6773115-9ef3-425f-bc65-a316372556ee None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.305157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.334953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.335627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.335902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.336171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.347069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.348501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.355514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611663a-f2fc-40fe-bf6c-7659caccd9dd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:15.363815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:15.364784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:15.365671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:15.365671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:15.365671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:15.369414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 431/862] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:15.370225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.399597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.400539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.400997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.401243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.401818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.403204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:15.413119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:15.414062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:15.414941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:15.420008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 432/863] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:15.420737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.422000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.474428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378423e-5ba8-4d4f-83a9-ebc83491441b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.509680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.510280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.510555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.510826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.511570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3a73ecd2399f434ba5999aa5be2f7fd9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:15.511876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.513285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.520145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 432/864] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/role_assignments?user.id=3a73ecd2399f434ba5999aa5be2f7fd9&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:15.520427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.554008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11473c18-c247-4f43-94cf-8a2ab80f49d2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.575542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.599786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.600582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.600929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.601275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.602446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:15.603066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.604801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.625734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 433/865] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:15.626826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.681290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5410618-4985-4e61-a65a-0caf36f254b5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.703892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.734584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.735227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.735504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.735769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.736851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:15.737155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.739306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.744648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 433/866] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:15.751165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.770362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e257d58-90fe-4228-9fc9-1e243511e346 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.813002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.813864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.814208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.814404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.822502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.823986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:15.839970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:15.840990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:15.842412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 434/867] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:15.851976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:15.888593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c56124-1bd1-4c20-8e0e-42d134bb511b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:15.894401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:15.940046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:15.941558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:15.942091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:15.942551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:15.943835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:15.944591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:15.946531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:15.954708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 434/868] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:15.972657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89a11a-b919-4095-9b40-2e6682c112d6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:16.016793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114501a-dd54-491f-a237-80400daea888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:16.017411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114501a-dd54-491f-a237-80400daea888 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:16.017686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114501a-dd54-491f-a237-80400daea888 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:16.018014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114501a-dd54-491f-a237-80400daea888 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:16.018789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114501a-dd54-491f-a237-80400daea888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:16.019165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114501a-dd54-491f-a237-80400daea888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:16.020690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114501a-dd54-491f-a237-80400daea888 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:16.039203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 435/869] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:16.041674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:16.103630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:16.193549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d51422-23d8-4ab6-a6f9-689e025fe496 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:16.228927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:16.229522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:16.229796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:16.230060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:16.231138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=eae392d38ade4665b2385bae93b3e688) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:16.231437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:16.232849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab83015-4c22-4117-bc5c-9ad1346f14a7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:16.239741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 435/870] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:16 2022] GET /identity/v3/role_assignments?user.id=eae392d38ade4665b2385bae93b3e688&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:16.930027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b49e9e-a01b-4ba0-b2d4-7ea128144e44 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:16.930551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b49e9e-a01b-4ba0-b2d4-7ea128144e44 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:16.930551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b49e9e-a01b-4ba0-b2d4-7ea128144e44 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:16.932174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 436/871] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:16.938054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abece0f-59df-48a8-b467-6f8eb93b2096 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:16.938450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abece0f-59df-48a8-b467-6f8eb93b2096 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:16.938637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abece0f-59df-48a8-b467-6f8eb93b2096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:16.957243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0abece0f-59df-48a8-b467-6f8eb93b2096 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:16.958432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0abece0f-59df-48a8-b467-6f8eb93b2096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:17.029236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 436/872] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:17.051529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b49e9e-a01b-4ba0-b2d4-7ea128144e44 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:17.154946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:17.158788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b49e9e-a01b-4ba0-b2d4-7ea128144e44 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:17.194329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:17.194852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:17.195166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:17.195432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:17.196061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:17.197414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a504009-81ba-4e6d-906f-d9d51cae0371 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:17.215997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 437/873] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:17 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:17.809389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f4fb-d62c-401d-8f86-1114318dabf1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:17.810089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f4fb-d62c-401d-8f86-1114318dabf1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:17.810597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588f4fb-d62c-401d-8f86-1114318dabf1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:17.813045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 437/874] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:17.818375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:17.818883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:17.819381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:17.837868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:17.839406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:17.917968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 438/875] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:17.942281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588f4fb-d62c-401d-8f86-1114318dabf1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:17.985482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.055963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588f4fb-d62c-401d-8f86-1114318dabf1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.107228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.107921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.108208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.108480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.119927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f5ec3712-0ee1-44e5-91a9-bec5138b44eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:18.124827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:18.125734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:18.126567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 438/876] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:18.133915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.223757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.243944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3396f6-17ab-432d-902a-04523443fce3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.278794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.279777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.280255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.280748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.282173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:18.282675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.284254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1bda86-d93e-4d4b-bc70-ec0766eaa8a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.292677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae529c-f443-41b1-84c7-60736fdeba07 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.293002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae529c-f443-41b1-84c7-60736fdeba07 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.293230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae529c-f443-41b1-84c7-60736fdeba07 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.294099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 439/877] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:18.299008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 439/878] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:18.300765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.301263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.301756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.302311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae529c-f443-41b1-84c7-60736fdeba07 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.320826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:18.322322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:18.396944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 440/879] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:18 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:18.414459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.423544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae529c-f443-41b1-84c7-60736fdeba07 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.425445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.462885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.463550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.463824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.464092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.475220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.476661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-212e9047-ce05-43a6-a8ee-74e35c311db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:18.492375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:18.493524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:18.494576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:18.494576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:18.494576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:18.496237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 440/880] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:18.497540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eefd1d-49a3-4470-b14e-32a430276119 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.497959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eefd1d-49a3-4470-b14e-32a430276119 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.498232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eefd1d-49a3-4470-b14e-32a430276119 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.499719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 441/881] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:18.501369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eefd1d-49a3-4470-b14e-32a430276119 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.516055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.543595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c4e22-a246-4b5d-ba30-230c770cc218 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.576962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.577493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.577772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.578572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.579254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.580612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c55d4a-5ec7-428f-919a-2db299316a45 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.596506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 441/882] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:18 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:44:18.597694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.598070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.598344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.617441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:18.618656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:18.635909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eefd1d-49a3-4470-b14e-32a430276119 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.686233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.687439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.687594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.687791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.688559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:18.688869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.690366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.704773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 442/883] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:18.710039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.714829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.739007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 442/884] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:18.755414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.829705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:18.839379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fe151-9453-4d14-a2a7-ba802166cf9f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.882704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e20cd-4b07-423e-83cf-8ae500351bfd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:18.886326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.887212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.887694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.888164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.898693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.901145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:18.916108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:18.917113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:18.918067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:18.920622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 443/885] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:18.927113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:18.927113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:18.927113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:18.927113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:18.927462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:18.927632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:18.927711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:18.945063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:18.946155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:18.947032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:18.948666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 443/886] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:18.955010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.001740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.048364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9d02-6deb-4889-a9bc-7a01ef3b5919 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.065357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402481d-6b73-476a-9694-5e21c2f32873 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.092814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.099455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.100453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.100875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.101247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.101473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.101846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.102123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.102399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.102791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.103044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.103350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.103553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.104706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.105272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.114619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 444/887] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:19.115537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.131114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 444/888] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:19.132977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.223505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc61249-bb78-4500-9907-41f215fd7d9e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.242993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2b98b-fd55-4f43-9c47-d84ddd81e1bf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.259729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.260387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.260662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.260933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.261610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.261913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.263517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29e168-5c33-45b2-aa38-0888f231c698 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.268138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 445/889] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:19.269291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b77e5e-86cb-47c6-9829-2c7bdcda6418 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.269572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b77e5e-86cb-47c6-9829-2c7bdcda6418 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.270115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b77e5e-86cb-47c6-9829-2c7bdcda6418 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.270233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.277522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 446/890] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:19.281255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.281665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.282066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.287606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.288228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.288501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.288766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.305591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:19.307692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:19.372038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.374062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.384847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.415006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 447/891] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:19.434623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.514780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 445/892] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:44:19.520913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.549243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.566204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa33d24-1251-4a54-9d86-88f4e828cbdc None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.603292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.604015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.604298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.604567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.605895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.606864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:19.616138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:19.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:19.618378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:19.621454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 448/893] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:19.631160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.645407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ebe41-6f39-4a2c-8438-a98d9cc3e262 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.685010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.685740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.686083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.686437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.697631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.699981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.701810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:19.726039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:19.727780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:19.728744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:19.728744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:19.728744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:19.728744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 446/894] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:19.735976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.763498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0ac1d-19cd-4521-88a0-42237beebf3c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.801069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.802095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.802561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.803096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.803946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.804466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.806030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.811490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 449/895] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:19.818042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.824005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.853004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5bba1-8244-455e-8e66-8d36ae058773 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.890958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.891514 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.891818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.892383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.892811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.893128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.894467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.911249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 447/896] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:19.913331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:19.937687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aae81-c9d1-417d-9d4c-0d640f709e22 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:19.953367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:19.976747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:19.977457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:19.977842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:19.978272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:19.978939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:19.979358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:19.981265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:19.992143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 450/897] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:19.993460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:20.026279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2d3ea-29d5-4ba3-a211-6f1eb7acac58 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:20.075772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:20.076690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.077161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.077711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.080103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:20.086278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:20.088159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:20.100374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:20.101291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:20.102174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:20.104341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c66137-fbd2-4f48-8cca-7f514a824a32 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:20.110075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 448/898] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:20.111162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:20.142830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:20.143421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.143697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.143965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.145077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] RBAC: Adding query filter params (interface=public, service_id=7a110b54bac246d7824e4dd87e5dcc26, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:20.145259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:20.147424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:20.163060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 451/899] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:19 2022] GET /identity/v3/endpoints?service_id=7a110b54bac246d7824e4dd87e5dcc26&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:20.165942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:20.214050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:20.242601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aae672-705e-472f-85b3-1104762dbf1a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:20.278918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a73ce0-f21a-4549-a539-4db2c28470d1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:20.286873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:20.287502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.287775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.288067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.289273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:20.289575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:20.291483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:20.314261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 449/900] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:20.315879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:20.324458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:20.325402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.325877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.326341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.327233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:20.328960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecacc23-b5ec-40d2-8bc2-2bdea0262e90 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:20.335630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 452/901] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:20 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:44:20.386577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:20.441142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce108-6e4e-493d-acf8-32a0dd9da099 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:20.482843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:20.483856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.484924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.485432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.486412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] RBAC: Adding query filter params (role.id=cffd93c2cc654062a35c70900c6f7ad8, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3a73ecd2399f434ba5999aa5be2f7fd9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:20.486944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:20.488623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd08d-fe46-4b2d-91d8-ef2a0e6c4322 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:20.499194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 450/902] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:20 2022] GET /identity/v3/role_assignments?user.id=3a73ecd2399f434ba5999aa5be2f7fd9&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=cffd93c2cc654062a35c70900c6f7ad8 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:20.773406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb41f-5b3a-4dd9-815d-956b598e9126 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.773783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb41f-5b3a-4dd9-815d-956b598e9126 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.774012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb41f-5b3a-4dd9-815d-956b598e9126 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.775670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 453/903] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:20.781674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:20.782058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:20.782373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:20.801027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:20.802235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:20.877632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 451/904] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:20.900557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beb41f-5b3a-4dd9-815d-956b598e9126 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.014026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beb41f-5b3a-4dd9-815d-956b598e9126 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.022840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:21.056062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.062506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.063103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.063590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.064399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.071861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:21.082545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:21.083591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:21.084481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:21.087340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 454/905] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 9 17:44:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:21.095041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.155391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:21.199770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56239e27-a035-4c48-92af-741d9bd9a382 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.235165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.235776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.236053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.236327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.236957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:21.237260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.238804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9fa87-4e6c-4ee9-b195-0d66c000d68a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.244045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 452/906] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:21.250150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.363462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e170-c44a-47d3-a044-3579477be227 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.399720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b5273-ee39-4e69-8ff7-4c5028c01bb3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.400080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b5273-ee39-4e69-8ff7-4c5028c01bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.400283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b5273-ee39-4e69-8ff7-4c5028c01bb3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.401042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.401820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.401936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 453/907] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:21.402344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.402620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.407092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:21.407092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.407092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.408712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.409204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.409706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.410354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 455/908] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:21.418460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.430217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:21.431745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:21.462641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:21.511246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 454/909] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:21.528903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.545522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7334-33eb-441d-b08e-f26a8eb736e2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.579869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:21.592594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.593780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.594533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.594810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.595839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] RBAC: Adding query filter params (interface=public, service_id=1bf473b0cbf04708bfffe2f7352f67f0, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:21.596151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.597799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.610603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 456/910] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:21 2022] GET /identity/v3/endpoints?service_id=1bf473b0cbf04708bfffe2f7352f67f0&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:21.612173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:21.652580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca67367-831d-43cd-a086-c7053a7d8a52 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.699269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.699889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.700158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.700432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.701362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.702894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196d33c-dd0c-4ea1-9088-4bcf394ecac7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.723032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 455/911] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:21 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:21.756801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:21.762764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e1252-99d6-4eb1-b149-8e717b48c743 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:21.822548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:21.824174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:21.825001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:21.825913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:21.827254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:21.830154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16932fac-e340-48bb-83bd-4f8d1dc2a04a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:21.839288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 457/912] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:21 2022] GET /identity/v3/services => generated 1497 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:44:22.486675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554b82e-63ac-45af-9071-83946be256c5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:22.487316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554b82e-63ac-45af-9071-83946be256c5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:22.487540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554b82e-63ac-45af-9071-83946be256c5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:22.489572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 456/913] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:22.496402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d706c-8592-47cf-9f40-c4e3d41ef7ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:22.496941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d706c-8592-47cf-9f40-c4e3d41ef7ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:22.497330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d706c-8592-47cf-9f40-c4e3d41ef7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:22.518213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-215d706c-8592-47cf-9f40-c4e3d41ef7ac None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:22.519555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-215d706c-8592-47cf-9f40-c4e3d41ef7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:22.598401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 458/914] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:22.620284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554b82e-63ac-45af-9071-83946be256c5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:22.677058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8201d9-ab51-4077-b48c-1bbc170d15a8 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:22.677395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8201d9-ab51-4077-b48c-1bbc170d15a8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:22.677610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8201d9-ab51-4077-b48c-1bbc170d15a8 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:22.679201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 459/915] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:22.686431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:22.686763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:22.687709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:22.707218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:22.708651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:22.750339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554b82e-63ac-45af-9071-83946be256c5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:22.752173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:22.791314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 460/916] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:22.792247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:22.792887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:22.793202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:22.793487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:22.794132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:22.795600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:22.806425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:22.807505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:22.808345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 457/917] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:22.814158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:22.821430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:22.839647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:22.947978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f0a60-142a-4ee8-9a72-f2c4984044f5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:22.971189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00cc87-26cd-4775-80c4-1eaad81f2700 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.003072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.004520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.004978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.005676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.006472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:23.007186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.009537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.013515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.014538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.014920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.015420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.016350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 461/918] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:23.021746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.030409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:23.037390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:23.038438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:23.039315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:23.042768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 458/919] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:23.045332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.079256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.121723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.143332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b991457-3bff-4a16-86a3-7b0662fefe8a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.165140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ecf7-4c8f-4fe0-be9d-ce3e410ecb1d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.182048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.182653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.182933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.183262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.184174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:23.184478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.186099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.193743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 462/920] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:23.198805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.210696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.212116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.212649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.213551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.214768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:23.215308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.217848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.232007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 459/921] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:23.237434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.315374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4071f06-9341-4b31-9aa7-8f1e3bbef280 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.356764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.357381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.357663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.358053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.358725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.361118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.364346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.379694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34121b9-f86f-4847-9485-ecc310abbcfd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.385727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 463/922] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:44:23.433370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.434504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.435067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.435503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.448602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.450432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:23.467658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:23.469521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:23.470629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:23.470629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:23.470629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:23.472632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 460/923] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:23.479181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.482000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.599345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8ab48-689d-4e67-95aa-0101b053328f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.637819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.638436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.638717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.639017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.639670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:23.640029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.641485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.653855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 464/924] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:23.660433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.772595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97bdd-e62e-43c7-b5af-fd27b63b8b37 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.809043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:23.809673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.809950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.810217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.813288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.819880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:23.821409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6a109140-df8c-40f7-bc94-a18be6d00cbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:23.839157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:23.840177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:23.841142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 461/925] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:23.847990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:23.933870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:23.960153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a902159-47f2-4f51-aa8c-076a806f09c6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.960469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a902159-47f2-4f51-aa8c-076a806f09c6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.960704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a902159-47f2-4f51-aa8c-076a806f09c6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.962321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 462/926] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:23.967535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb7e8e-03bb-4faf-92ac-c9131e543157 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:23.967810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb7e8e-03bb-4faf-92ac-c9131e543157 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:23.968089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb7e8e-03bb-4faf-92ac-c9131e543157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:23.985060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa4a-5167-4a19-b424-97b8c5982b18 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:23.989088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-53bb7e8e-03bb-4faf-92ac-c9131e543157 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:23.990380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53bb7e8e-03bb-4faf-92ac-c9131e543157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:24.029213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.030516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.030516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.030516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.031699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.032003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.033419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd0d00-bfd6-4d64-b075-29cb1902c84f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.054897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 465/927] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.054897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644248a-f562-4162-8c15-355d5208c8a1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.054897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644248a-f562-4162-8c15-355d5208c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.054897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644248a-f562-4162-8c15-355d5208c8a1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.054897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 466/928] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:24.059844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644248a-f562-4162-8c15-355d5208c8a1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.079007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 463/929] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:24.080302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.080527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.080801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.099968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:24.101265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:24.172271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 464/930] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:24.173395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.187924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644248a-f562-4162-8c15-355d5208c8a1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.210126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.226883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.227544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.227824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.228093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.236813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.240082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.241730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:24.259006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:24.259991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:24.260869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:24.260869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:24.260869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:24.261036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 467/931] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:24.261801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.291109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96e894-6f76-4f94-bd8f-672bedb41f79 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.328696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.329325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.329606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.329875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.330569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.332163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:24.341224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:24.342157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:24.343075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 465/932] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:24.344058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.375256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298ea6-31ce-4d65-a419-4b5304869d89 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.412765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.413439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.413688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.413957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.414530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.415915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:24.425298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:24.426305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:24.427169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 468/933] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:24.427960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.460085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002c6a0-a9eb-4c30-8316-c22abc21ac49 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.473587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.482785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.498450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.499096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.499376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.500282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.500973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.501553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.502669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.514875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 466/934] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.520125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.550990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b1fa1-d84a-4d41-a8a0-dc5ed2677487 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.587888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.588492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.588768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.589298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.589661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.589961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.591341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.601771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 469/935] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.601771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.636539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acef381-3301-46ec-afdc-a25ad4ea604a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.676898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.677571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.677774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.678046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.678681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.679003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.680382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.685316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 467/936] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.686075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.720093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.723075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d0409-0413-4439-b15a-2f52854a8010 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.741843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:24.763306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.763929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.764211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.764515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.771281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.772714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:24.783898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:24.785384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 470/937] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:24.786269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.823136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836775e-7b59-4f44-8bab-748208ed6128 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.864020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.864860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.865289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.865698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.867011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.867419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.868905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.878848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 468/938] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.878848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:24.941088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2b98-348c-47b3-8e48-33f3785a4042 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:24.983115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:24.985951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:24.985951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:24.985951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:24.985951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:24.985951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:24.987212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:24.995199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 471/939] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:24.995199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.012898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52565-92b4-4a67-959a-bfb935ac3b2d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.024622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:25.026388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:25.065758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.067235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.067235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.067847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.068374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:25.068775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.070326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.084169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 469/940] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:25.085596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.126965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228d5c6-9efb-47b5-b3dd-854fac284b30 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.166103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.166729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.167024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.167299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.167953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.169430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.190844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb801f3-d59d-4c85-84e1-c01a9d26f980 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.203008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 472/941] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:25.203008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.254168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.254825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.255205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.255552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.256865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] RBAC: Adding query filter params (interface=public, service_id=f4af3e121151406b8c960b655954b3c2, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:25.257257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.259606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.268287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 470/942] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:25 2022] GET /identity/v3/endpoints?service_id=f4af3e121151406b8c960b655954b3c2&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:25.274895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.309250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:25.318962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:25.344968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f3625-bf53-4929-824e-11e47854b0f8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.396324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.398562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.399881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.400397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3a73ecd2399f434ba5999aa5be2f7fd9/roles/cffd93c2cc654062a35c70900c6f7ad8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.407725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=3a73ecd2399f434ba5999aa5be2f7fd9, role_id=cffd93c2cc654062a35c70900c6f7ad8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.409525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e90da-ad8a-41b8-a373-febf575af774 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.419459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3fb557-b592-4c17-9176-73702c155639 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.425127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 473/943] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:25 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/3a73ecd2399f434ba5999aa5be2f7fd9/roles/cffd93c2cc654062a35c70900c6f7ad8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:25.464158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.464781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.465058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.465322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.465933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.467353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36d881-fb76-4c12-8dda-ffbcc86bc64b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.475265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 471/944] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:25 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:44:25.560207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b25c-1658-4c5f-915c-624a63a3df33 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.560403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b25c-1658-4c5f-915c-624a63a3df33 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.560690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24b25c-1658-4c5f-915c-624a63a3df33 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.562736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 474/945] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:25.569645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.569932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.570251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.591631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:25.592897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:25.697341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 472/946] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:25.721247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b25c-1658-4c5f-915c-624a63a3df33 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.850550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24b25c-1658-4c5f-915c-624a63a3df33 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:25.885794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:25.886394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:25.886666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:25.886932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:25.887589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:25.889009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:25.892207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING keystone.server.flask.common [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 17:44:25.902800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:25.911590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.sql.core [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:44:25.911590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 17:44:25.911590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [parameters: {'id': '0c37eb1e4bcb4465b189ba67d0b6c9fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 17:44:25.911590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c37eb1e4bcb4465b189ba67d0b6c9fe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c37eb1e4bcb4465b189ba67d0b6c9fe. Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:44:25.930242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 17:44:25.931148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application yield resource Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise self.value Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application session.commit() Aug 09 17:44:25.932090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:44:25.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:44:25.933856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:44:25.934624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [parameters: {'id': '0c37eb1e4bcb4465b189ba67d0b6c9fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:25.938170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:25.940098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c37eb1e4bcb4465b189ba67d0b6c9fe. Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:25.947732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 475/947] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Aug 9 17:44:25 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:25.949030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:25.965490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:26.052487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21db692-1372-4050-9c29-c95fcd5bc3a2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:26.092449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:26.093069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:26.093431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:26.093698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:26.100717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:26.102168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8555b1cf-69e5-40e9-9dcd-21c2a5eafe59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:26.114432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:26.116624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:26.118736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:26.119788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 473/948] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:26.124322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:26.237282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e6c0-accc-4840-89e4-2b30e64e17ee None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:26.272909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:26.273512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:26.273780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:26.274046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:26.274706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:26.275026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:26.276624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9443e48-8bc4-4bd1-9a70-07cf81d13ac5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:26.297229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 476/949] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:27.504530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e645e-0674-4b6c-860a-c4c7452a54ca None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.505251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e645e-0674-4b6c-860a-c4c7452a54ca None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.505251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e645e-0674-4b6c-860a-c4c7452a54ca None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.506718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 474/950] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:27.512274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.512800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.512800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.531530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:27.533019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:27.547350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad114cd-227b-4ff6-bf04-073c72fe1cdf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.547680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad114cd-227b-4ff6-bf04-073c72fe1cdf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.547937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad114cd-227b-4ff6-bf04-073c72fe1cdf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.549284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 475/951] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:27.554403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6024-0a92-4dc7-a637-6446f45de887 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.554686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6024-0a92-4dc7-a637-6446f45de887 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.554962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e6024-0a92-4dc7-a637-6446f45de887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.573459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3e3e6024-0a92-4dc7-a637-6446f45de887 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:27.574648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e3e6024-0a92-4dc7-a637-6446f45de887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:27.593472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:27.617323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 477/952] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:27.644394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:27.665870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 476/953] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:27.688672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ef5a-a8e8-471b-83a0-9a0cb78e9abb None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.688989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ef5a-a8e8-471b-83a0-9a0cb78e9abb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.689816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5ef5a-a8e8-471b-83a0-9a0cb78e9abb None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.691849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 477/954] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:27.692506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5ef5a-a8e8-471b-83a0-9a0cb78e9abb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:27.707813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:27.754044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23eef3-982d-4fae-b6c5-e3aa5d22c9f2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:27.791352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:27.791898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.792168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.792433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.793008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:27.795088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-17810dfb-fffc-46bc-9a64-b8fecce24bb5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:27.806627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:27.807969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:27.808876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:27.809666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5ef5a-a8e8-471b-83a0-9a0cb78e9abb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:27.809742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 478/955] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:27.809742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.809742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.809742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.828898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:27.830072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:27.848963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:27.849749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:27.849947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:27.850088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:27.862209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:27.867313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:27.868296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 478/956] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:27.874452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:27.882179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:27.906335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 479/957] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:27.907274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:27.958318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:27.995782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984edd5-2f3c-4200-84aa-d783c3f8415a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.018522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8191a4a-857b-47c9-a3b1-e30ff57a452b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.032038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.032647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.032925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.033225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.033824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.034309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.035524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.040819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 479/958] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.041658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.071557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.072241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.072492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.072713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.073379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.073676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.075115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.084294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 480/959] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.084989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.131165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.157501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500d6f-0e1c-4699-a24c-28e516fe138a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.195488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.196004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c409d90a-a3f0-491f-9f16-d44ce64c3a5a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.196217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.196455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.196655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.197240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.198609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.204098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:28.210921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:28.212211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:28.213263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:28.216287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 480/960] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:28.217086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.238678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.239354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.239633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.239889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.240506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.240804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.242147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.247070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 481/961] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.247854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.362574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fadd4-6c02-4e2a-9c79-2ee6d223dadf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.388064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9deb46-a32f-4cab-9c65-b38b53f64542 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.397650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.414089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.415665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.415665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.415878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.428739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.430217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.435774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.436388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.436664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.436999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.438078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.438383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.440227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bc08a-2254-4f1e-b2bf-d11be24a1ebb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.445275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 482/962] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9566fce3-921e-4439-9758-3f5f59a0dc57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:28.445664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:28.446562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:28.447491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:28.447491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:28.447491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:28.447680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013bba-11eb-4c8f-9fe4-ec29a1910568 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.447680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013bba-11eb-4c8f-9fe4-ec29a1910568 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.447680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013bba-11eb-4c8f-9fe4-ec29a1910568 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.448488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 483/963] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:28.448710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 481/964] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:28.451065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013bba-11eb-4c8f-9fe4-ec29a1910568 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.456926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.457212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.457485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.475656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:28.476837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:28.528907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.550724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 482/965] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:28.551837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.576410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013bba-11eb-4c8f-9fe4-ec29a1910568 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.613759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.614282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.614558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.614827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.615488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.615789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.617197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.627749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 484/966] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.627749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.662178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.667217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717e39-054f-4c2f-a591-61bae9362456 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.706538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.707655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.707655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.707837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.708363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:28.708668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.710094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.726987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 483/967] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:28.726987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.739408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e805da-b228-4b9b-9033-63ce69a35692 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.776159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.776760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.777231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.777342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.777613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.778595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.780465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.796494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.sql.core [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:44:28.796494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 17:44:28.796494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [parameters: {'user_id': 'dff7430dd7734d9184fb8a678a6527ef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 17:44:28.796494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80212) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Aug 09 17:44:28.803483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:44:28.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application yield resource Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application next(self.gen) Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise self.value Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application session.commit() Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 17:44:28.805389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.session.flush() Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application compat.raise_( Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 17:44:28.806274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 17:44:28.807205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application conn.query(q) Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application result.read() Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 17:44:28.810180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dff7430dd7734d9184fb8a678a6527ef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:28.812959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 17:44:28.815990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 485/968] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 9 17:44:28 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:28.819296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.847160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325845da-99ea-4e46-94d3-cf4c30761e08 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:28.887149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:28.887989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:28.888473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:28.888911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:28.889856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:28.891998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:28.913976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 484/969] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:28.915145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:28.933426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:28.947578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86571d7-f70a-4f35-834d-2d13fe469109 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.010692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.011874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.013350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.015011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.025757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.028006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:29.041416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:29.042439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 17:44:29.043525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:29.047012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 486/970] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:29.053214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:29.088509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:29.094936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c8fc-9a09-461c-be43-35bc72f930c1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.145304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.146187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.146452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.146727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.156410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.157836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:29.173127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:29.174323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:29.175554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 485/971] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:29.180907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:29.193912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49ca03-d217-4d70-b9c9-5b8119ccda47 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.229687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.230407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.231040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.231671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.232250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:29.232624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.234657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.262340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 487/972] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 9 17:44:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:29.263552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:29.278276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:29.298946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523de0c7-7f83-4aa3-bf57-74315d410c94 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.338171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.338823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.339193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.339434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.340147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:29.340445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.341889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.387000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 486/973] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 9 17:44:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:29.403759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c99ba6-b735-4e11-9e0d-ea1c41424b45 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.448705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.449474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.449829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.450172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.530085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.532262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.547900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:29.655458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 488/974] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:29 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:29.662465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:29.683993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:29.777886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640db3d6-b0b6-45d0-a7fd-bc76ce63dc24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:29.817536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:29.818184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:29.818471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:29.818740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:29.831305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:29.832842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:29.851660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:29.852689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:29.853589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:29.853589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:29.853589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:29.855051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 487/975] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:29.858881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:29.994820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed725426-05a3-4554-98a6-9ab4ceb67712 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:30.011087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:30.038500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:30.039237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.039580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.039912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.040831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:30.041211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:30.042704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:30.062166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 489/976] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:30.063815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:30.149232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:30.193726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eb1a-b00c-4789-a6c9-7029f1afe973 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:30.235736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:30.236402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.236681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.236955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.245372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:30.246802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-466b2040-69b1-456d-bf79-e678d44eb0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:30.260532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:30.262278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:30.263440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 488/977] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:30.269457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:30.381984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297740c-068a-4df8-b9ae-0a2589c9ca80 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.382267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297740c-068a-4df8-b9ae-0a2589c9ca80 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.382555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297740c-068a-4df8-b9ae-0a2589c9ca80 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.383899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 489/978] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:30.388720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.388998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.389263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.394279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a66ca-6e8a-4748-9e8b-6ec2315b8fdd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:30.409261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:30.410596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:30.440053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:30.440691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.441033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.441308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.442018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:30.442323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:30.443825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:30.457837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 490/979] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:30.464684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:30.497992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 490/980] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:30.503813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:30.523301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:30.586892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8f92-0670-4e45-ac89-f797d54c1a6e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:30.605328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:30.626237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:30.626847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.627211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.627483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.628210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] RBAC: Adding query filter params (role.id=cffd93c2cc654062a35c70900c6f7ad8, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=3a73ecd2399f434ba5999aa5be2f7fd9) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:30.628516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:30.630064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbfd6-d699-40a5-abf0-dd09740668e4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:30.637280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 491/981] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:30 2022] GET /identity/v3/role_assignments?user.id=3a73ecd2399f434ba5999aa5be2f7fd9&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=cffd93c2cc654062a35c70900c6f7ad8 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:30.661508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd2677-6ef5-4cfb-b159-24709cc903d5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:30.713827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:30.714621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:30.714806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:30.714998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:30.715691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:30.717114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec29f1-4588-4afa-855d-58ef73082c9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:30.747026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 491/982] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:30 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:44:31.128887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08059fdc-3ec6-4247-9f67-2259b7bfa0f1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.129413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08059fdc-3ec6-4247-9f67-2259b7bfa0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.129413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08059fdc-3ec6-4247-9f67-2259b7bfa0f1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.130887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 492/983] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:31.135913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005742-d747-49cb-b858-8499427d9093 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.136202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005742-d747-49cb-b858-8499427d9093 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.136472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005742-d747-49cb-b858-8499427d9093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.154299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-62005742-d747-49cb-b858-8499427d9093 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:31.156042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-62005742-d747-49cb-b858-8499427d9093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:31.234317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 492/984] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:31.256917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08059fdc-3ec6-4247-9f67-2259b7bfa0f1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:31.353652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:31.362632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08059fdc-3ec6-4247-9f67-2259b7bfa0f1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:31.396740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:31.397506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.397878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.398240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.398903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:31.400378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:31.409678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:31.410846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:31.411726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 493/985] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:31.420087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005742-d747-49cb-b858-8499427d9093 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:31.467667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:31.528111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005742-d747-49cb-b858-8499427d9093 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:31.561918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:31.562519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.562804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.563109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.563734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:31.564036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:31.565395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:31.570396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 493/986] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:31.576047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:31.680347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddcf68-04dd-4bfb-ac31-f6c4efbbd118 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:31.714666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:31.715302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.715582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.715852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.716480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:31.716781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:31.718162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:31.723321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 494/987] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:31.729185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:31.836403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b578f-0af6-41cd-bc79-02d0ec76ae20 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:31.846263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:31.873773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:31.874428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:31.874708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:31.875066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:31.875742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] RBAC: Adding query filter params (interface=admin, service_id=f4af3e121151406b8c960b655954b3c2, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:31.876047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:31.877423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac1081-c0f4-4d2e-8f43-2763d94aaa5b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:31.888834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 494/988] 10.222.0.52 () {56 vars in 1249 bytes} [Tue Aug 9 17:44:31 2022] GET /identity/v3/endpoints?service_id=f4af3e121151406b8c960b655954b3c2&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:31.890135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:31.915968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:31.993743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c369a-514e-4f7d-a64b-5f271ae40fd0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:32.028258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:32.028791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:32.029071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:32.029343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:32.029945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:32.031934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169c10e-70de-4beb-94ca-5152d8ba6e19 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:32.038803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 495/989] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:31 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:44:32.786652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f463e1-1e53-4a2f-abd9-edaafc294669 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:32.787251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f463e1-1e53-4a2f-abd9-edaafc294669 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:32.787251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f463e1-1e53-4a2f-abd9-edaafc294669 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:32.788746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 495/990] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:32.793630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:32.794030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:32.794190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:32.813507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:32.815031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:32.842012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93507-714c-4b5c-87a9-23bf7b23dd40 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:32.842295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93507-714c-4b5c-87a9-23bf7b23dd40 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:32.842576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93507-714c-4b5c-87a9-23bf7b23dd40 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:32.844107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 496/991] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:32.848875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:32.849166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:32.849441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:32.867757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:32.868947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:32.886386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 496/992] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:32.908906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:32.927293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:32.941706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 497/993] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:32.963744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.021215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f6855-da9c-4676-8e67-5a05c5568ea7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.066722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.067310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.067675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.067950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] PATH_INFO: `/v3/services/image` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.068574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.069885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.075792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e61195-0a01-46de-a091-8d040b2ed467 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.077329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:33.080371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:33.081381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:33.082683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 497/994] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 9 17:44:32 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:33.087212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.120107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.120737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.121015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.121276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.121861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.123272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:33.132829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:33.133925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:33.134745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 498/995] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:33.140019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.181342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:33.202595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886cd7-2a2f-454b-b7a1-4a5aa68f8292 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.238958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.239597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.239878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.240150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.240776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] RBAC: Adding query filter params (name=image) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:33.241080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.242448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.247239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 498/996] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:33.252722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.261042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0b363-e0c9-491d-b588-cb6eedaa9a0a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.299795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.300630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.300932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.301262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.302079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:33.302381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.303761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.310825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 499/997] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:33.316385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.347372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:33.365614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41509181-592a-43c1-b452-382ac5b7dcdb None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.402408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.403150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.403425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.403690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.404305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] RBAC: Adding query filter params (type=image) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:33.404599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.406698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.412116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 499/998] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:33.435051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269bc0d-ea8c-45e7-b6e0-4f57e4e9aa2b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.473907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.474851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.475331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.475757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.476587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:33.477129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.478729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.485871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 500/999] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:33.491825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.504103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:33.606349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745785-58a6-439b-a716-b67ed3f19fab None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.643508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.644123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.644396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.644658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.645355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] RBAC: Adding query filter params (interface=public, service_id=adf9535f795d447a93556f784895b79f, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:33.645657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.647024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69304c-6d68-4046-94a1-1c3b03ca1d77 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.652854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 500/1000] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/endpoints?service_id=adf9535f795d447a93556f784895b79f&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:33.658670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:33.754952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:33.762497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de275c-5c9b-4a60-aa2b-777b1204f543 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:33.802431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:33.804623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:33.804921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:33.805201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:33.806821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:33.808288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5ae11-1e72-4e7f-a77d-a1b542ee9551 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:33.815654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 501/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:33 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:44:34.059192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a3538-c1dd-41bb-b733-2aa91b4b2248 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.059192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a3538-c1dd-41bb-b733-2aa91b4b2248 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.059192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a3538-c1dd-41bb-b733-2aa91b4b2248 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.059824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 501/1002] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:34.067135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.067135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.067135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.085953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:34.087192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:34.157646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 502/1003] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:34.179476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a3538-c1dd-41bb-b733-2aa91b4b2248 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:34.255834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:34.290724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a3538-c1dd-41bb-b733-2aa91b4b2248 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:34.326238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:34.326780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.327079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.327432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.328008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:34.329415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:34.338489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:34.339528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:34.344023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 502/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 9 17:44:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:34.347771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:34.438651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:34.451719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97892f29-1e67-4cb2-8677-76344eafe969 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:34.487711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:34.488320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.488593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.488859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.489486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:34.489784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:34.491175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:34.496852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 503/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:34.502943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:34.618361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af014f5e-6d99-436e-b040-6020ff687b3a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:34.652655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:34.653246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.653517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.653780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.654390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:34.654683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:34.656628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e37b5-5f17-47d6-9fe4-85c471f3fc29 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:34.658730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:34.667115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 503/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 9 17:44:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:34.668720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:34.772164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4485468-541b-4ff3-a6c4-03bbe5606f1a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:34.808883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:34.809541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:34.809756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:34.810027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:34.810649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:34.812076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8592c3bf-4d9c-4303-8484-759008687991 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:34.834996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 504/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:35.461800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741b6c-76d4-44c8-befc-6cc0902ba20a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:35.462347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741b6c-76d4-44c8-befc-6cc0902ba20a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:35.462347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40741b6c-76d4-44c8-befc-6cc0902ba20a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:35.463938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 504/1008] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:35.469041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfc482-0804-4307-9e50-2cf4056a4cc2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:35.469380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfc482-0804-4307-9e50-2cf4056a4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:35.469642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cfc482-0804-4307-9e50-2cf4056a4cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:35.488602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-29cfc482-0804-4307-9e50-2cf4056a4cc2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:35.489770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-29cfc482-0804-4307-9e50-2cf4056a4cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:35.497907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:35.568802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 505/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:35 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:35.590616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741b6c-76d4-44c8-befc-6cc0902ba20a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:35.699012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40741b6c-76d4-44c8-befc-6cc0902ba20a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:35.734773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:35.735315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:35.735585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:35.735851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:35.736488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:35.737839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2189452-6751-4c5b-89e4-bdb9e612363d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:35.760515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 505/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:35 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:35.909397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475292-324b-42db-b7c4-aa9f21f7471d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:35.909955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475292-324b-42db-b7c4-aa9f21f7471d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:35.909955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475292-324b-42db-b7c4-aa9f21f7471d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:35.911501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 506/1011] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:35.916546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:35.916830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:35.917097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:35.937320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:35.938500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:35.999986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:36.019951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 506/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:35 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:36.041794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475292-324b-42db-b7c4-aa9f21f7471d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:36.157178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475292-324b-42db-b7c4-aa9f21f7471d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:36.192441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:36.193065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.193343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.193615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.194299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:36.195666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:36.204605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:36.205658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 17:44:36.206551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:36.210687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 507/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:36.211896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:36.314140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a3ae3-1131-4dc2-a76d-7a66f093d871 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:36.349236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:36.349848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.350239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.350512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.351171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:36.351473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:36.352837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:36.358428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 507/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:36.364419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:36.367256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:36.472428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89db297-1db7-4d79-b1a5-1cede296281f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:36.509961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:36.510572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.510851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.511153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.511794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] RBAC: Adding query filter params (type=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:36.512101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:36.513465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8017df6-1332-4594-b1d7-8987f18f6dda None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:36.523547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 508/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 9 17:44:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:36.525113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:36.577049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:36.629286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9d29d-6e98-4407-9321-99b7825b77d8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:36.669068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:36.670078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.670555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.671068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.671923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:36.673538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd7987-5def-4986-b9b4-1cbc523f9aa0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:36.694665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 508/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:36.926780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c21ea7-c59f-43bb-9721-e3d8303d17e0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.927357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c21ea7-c59f-43bb-9721-e3d8303d17e0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.927357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c21ea7-c59f-43bb-9721-e3d8303d17e0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.928695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 509/1017] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:36.934369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:36.934652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:36.934932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:36.955520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:36.957141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:37.042580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 509/1018] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:37.069972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c21ea7-c59f-43bb-9721-e3d8303d17e0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:37.142573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:37.181452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c21ea7-c59f-43bb-9721-e3d8303d17e0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:37.222239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:37.222873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:37.223140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:37.223349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:37.224462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:37.225894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260509-f9f6-4363-9642-b0ed7a6d2fc6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:37.231041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 510/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:37.234563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:37.269665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:37.345372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1eecc-7233-48df-ab2f-6785ab47d32f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:37.384776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:37.385406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:37.385681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:37.385947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:37.386588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:37.388057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e82f7-72e3-49d4-bd68-1a18235802dc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:37.415794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 510/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:44:37 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:37.842629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86365f8-a795-43e1-9fff-c5e7278165ba None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:37.843493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86365f8-a795-43e1-9fff-c5e7278165ba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:37.843726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86365f8-a795-43e1-9fff-c5e7278165ba None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:37.847027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 511/1021] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:37.850331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:37.850771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:37.850936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:37.870095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:37.871574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:37.945546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 511/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:37.969041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86365f8-a795-43e1-9fff-c5e7278165ba None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:38.080804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86365f8-a795-43e1-9fff-c5e7278165ba None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:38.115443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:38.116063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.116358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.116627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.117199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:38.118628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:38.120729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:38.128709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:38.129828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:38.130686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:38.135080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 512/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 9 17:44:37 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:38.137442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:38.154006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:38.240907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742a46-7227-444f-8e6d-dcab3e0799fb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:38.274059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:38.274649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.274925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.275217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.275838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] RBAC: Adding query filter params (name=image) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:38.276128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:38.277519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:38.284540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 512/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:38.290006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:38.393724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5e172-92c1-45db-8459-9d2f04bcf511 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:38.428155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:38.428750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.429023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.429287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.429907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:38.430204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:38.431651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:38.441907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 513/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:38.443632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:38.525639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:38.550291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2619e61-99e6-4112-95d5-ac96e7b2d315 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:38.587581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:38.588179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.588450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.588712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.589402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] RBAC: Adding query filter params (interface=public, service_id=88b2b675bb324db794e70360e6614cc5, region_id=RegionOne) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:38.589702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:38.591101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885ef05-0f73-4bd5-966f-9aeea1afd7db None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:38.601211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 513/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:44:38 2022] GET /identity/v3/endpoints?service_id=88b2b675bb324db794e70360e6614cc5&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:38.602815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:38.669886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:38.711955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1b3d2-11f4-412e-8fa9-fab3f79c3126 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:38.746883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:38.747515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.747799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.748069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.748672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:38.750062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d40df-65fa-48e8-94e1-d66c22b1400f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:38.756987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 514/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:44:38 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:44:38.834788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0aadd-af8b-4798-a8c5-5f90de31a3dd None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.835162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0aadd-af8b-4798-a8c5-5f90de31a3dd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.835634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0aadd-af8b-4798-a8c5-5f90de31a3dd None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.838062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 514/1028] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:38.844169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:38.844462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:38.844778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:38.865454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:38.866673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:38.938403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 515/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:38.962188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0aadd-af8b-4798-a8c5-5f90de31a3dd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.088390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0aadd-af8b-4798-a8c5-5f90de31a3dd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.125130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.125755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.126034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.126304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.137213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:39.142563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:39.143577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:39.144408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 515/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:39.151056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.219140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.266750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3c0f3-54a5-47e5-bbad-7dac76d3f8fc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.301351 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.301960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.302237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.302513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.303228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:39.303533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.304912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb6d83-d543-4aa3-8dcf-3f285aa89796 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.313878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 516/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:39.319646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.323079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.423245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10362699-6cd6-4587-b80f-b00d0642f527 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.457993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.458623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.458897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.459192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.471075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.472646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:39.488498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:39.489499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:39.490391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:39.490391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:39.490391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:39.491644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 516/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:39.492924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385226ce-55e5-48cc-ae81-8edb78d9a776 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.493216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385226ce-55e5-48cc-ae81-8edb78d9a776 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.493488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385226ce-55e5-48cc-ae81-8edb78d9a776 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.495221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 517/1033] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:39.495845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.500461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.500735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.500993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.519822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:39.521041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:39.558037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.598665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 518/1034] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:39.608103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bf7b6-08ae-4d17-8ace-8ca57d533271 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.622696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.645348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.645954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.646231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.646497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.647226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:39.647529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.648908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.661029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 517/1035] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:39.666541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.675097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.736837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569676ae-29ab-422e-8017-3d0c174eb79d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.773776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abca82-a6c6-453d-afb8-d6f334d09e86 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.777425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.778146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.778492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.778832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.780040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.781623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.783312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 519/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:39.789230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.789602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.815050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.815680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.815962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.816235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.823853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.825294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:39.838881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:39.840009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:39.840996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 518/1037] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:39.847519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:39.916057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f9090-2c8a-4f5f-9b34-e0932e075c14 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.934035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:39.957284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:39.958238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:39.958701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:39.959206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:39.960033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:39.961713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:39.966333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19624b-fab0-4d2b-bfed-facfd0f49007 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:39.987766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 520/1038] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:44:39 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:40.011700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:40.012626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.013009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.013284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.014073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:40.014381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:40.016482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:40.036902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 519/1039] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:44:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:40.042624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.159991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbe0da-a44f-4b97-887a-8ea95e3e0ba6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:40.197187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:40.197795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.198097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.198366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.253082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:40.282177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:40.283671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:40.396573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 521/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:40.397843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.430005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:40.504501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e395d-0a7e-4518-b649-c7509bf332fc None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:40.540506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:40.541514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.542018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.542480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.553927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:40.555676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:40.572622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:40.573706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:40.574642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:40.574642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:40.574642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:40.580067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 520/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:40.581324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.658463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:40.691070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab666-25e6-42be-a879-922bd9934a57 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:40.727038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:40.727646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.727925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.728196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.728846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:40.729146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:40.730774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932735-9627-4373-b382-06a5cf5c3fd6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:40.748135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 522/1042] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:40.749314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.788396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88879376-2013-4d74-a016-d6633353e24e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.788580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88879376-2013-4d74-a016-d6633353e24e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.792980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88879376-2013-4d74-a016-d6633353e24e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.792980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 523/1043] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:40.796786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.797523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.797862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.818715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:40.820029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:40.840232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:40.857306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8eeecd-a434-41c2-a244-1975e2d265f8 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:40.894011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:40.894736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:40.895112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:40.895444 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:40.899136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 524/1044] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:40.904284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:40.906253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:40.917120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:40.918069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:40.918965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 521/1045] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:40.921595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.925208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:40.986239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:41.028594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f619a6f-e66d-483c-9d43-edaa38139f87 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.040670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a2405-8ae8-4442-85d1-c26322aac1c1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.068560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.069200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.069475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.069742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.070392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.072041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:41.082153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:41.083233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:41.088236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 525/1046] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:41.090097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:41.096698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.097458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.097771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.098074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.098816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:41.099148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.100582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.115245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 522/1047] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:41.122575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:41.128256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:41.228305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacc4a3-53c5-4b4a-8241-83bd76b430cf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.259560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:41.264188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1f76b-6187-4bde-8553-b6011df4709d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.272741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.273441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.273717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.273993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.274623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] RBAC: Adding query filter params (name=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:41.274924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.276364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.288435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 526/1048] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:41.290531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:41.312076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.312601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.312880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.313144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.313873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] RBAC: Adding query filter params (role.id=cffd93c2cc654062a35c70900c6f7ad8, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:41.314171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.315705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.331370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 523/1049] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:41 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=cffd93c2cc654062a35c70900c6f7ad8 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:41.410129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9267b-dce6-43d6-ad2c-56c9eb16ba48 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.447011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.447621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.447902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.448177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.448823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:41.449131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.450538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d968c-3a7d-4488-a11e-9f8c521f4d9e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.461151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 527/1050] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:41.463094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:41.534904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:41.570801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ebed-5b83-4bce-959d-9fd6256ca87a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:41.606952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:41.607580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:41.607855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:41.608123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:41.609394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:41.610754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b7489-bcec-4f12-a173-88e5b8082c4f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:41.639005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 524/1051] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:44:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:42.107010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e6fb-f701-4be9-a813-c60398d9b43d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.107353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e6fb-f701-4be9-a813-c60398d9b43d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.107565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6e6fb-f701-4be9-a813-c60398d9b43d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.109383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 528/1052] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:42.114569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.115229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.115229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.134830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:42.136041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:42.213218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 525/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:42.238861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6e6fb-f701-4be9-a813-c60398d9b43d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:42.303954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:42.378227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6e6fb-f701-4be9-a813-c60398d9b43d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:42.420651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:42.421337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.421615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.421880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.433254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:42.438523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:42.439659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:42.440517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:42.443050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 529/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:42.445718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:42.497402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:42.548937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0797cb3-482b-4c0a-aaee-e4eafe4e2a3a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:42.585198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:42.585794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.586066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.586332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.587007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:42.587305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:42.588650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:42.601354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 526/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:42.603056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:42.705413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708984b-2a3a-4083-8738-16715fdc7b0b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:42.720104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:42.743103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:42.743761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.744043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.744344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] PATH_INFO: `/v3/users/819271fd689f4ee2a2212493bee2564d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.759356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] RBAC: Authorizing `identity:get_user(user_id=819271fd689f4ee2a2212493bee2564d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:42.760818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:42.767286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 530/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:42 2022] GET /identity/v3/users/819271fd689f4ee2a2212493bee2564d => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:42.768610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:42.875711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7fb73-98be-470f-acda-b030a144b3de None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:42.888533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:42.912038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:42.912734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:42.913085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:42.913424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:42.982389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:42.984434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:43.097428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 527/1057] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:43.103357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.207887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b2365-bde6-4899-ab93-e4a925e90f5c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.243858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.244495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.244752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.244932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.246032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:43.247489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4114be-29a1-40e7-8339-5f3e929643c2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:43.254040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 531/1058] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:43.255613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.270543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:43.348268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4603f5-f70b-4ae0-91bd-5bedc1ff5721 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.348565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4603f5-f70b-4ae0-91bd-5bedc1ff5721 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.348815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4603f5-f70b-4ae0-91bd-5bedc1ff5721 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.350251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 532/1059] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:43.355085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.355614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.355893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.359742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f9cbc-5d38-498c-a3eb-d672e779a909 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.373646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:43.374863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:43.397734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.398331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.398643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.398873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.399632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=d1a948696eea46958d179ab44c1b0ea3, user.id=819271fd689f4ee2a2212493bee2564d) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:43.399942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:43.401354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:43.407544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 528/1060] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/role_assignments?user.id=819271fd689f4ee2a2212493bee2564d&scope.project.id=d1a948696eea46958d179ab44c1b0ea3&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:43.415055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:43.454451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 533/1061] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:43.477304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.561755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:43.590493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822cc3-7728-404e-9a52-222eb3d8a3ee None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.626685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.627489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.627870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.628434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.640353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ad048c0c-d66d-48f6-a342-34293f929df8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:43.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:43.646263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:43.647319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:43.654065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 529/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:43.655251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.688644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d14e1-3138-40ab-8f75-25a6c205a2ed None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.688918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d14e1-3138-40ab-8f75-25a6c205a2ed None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.689188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d14e1-3138-40ab-8f75-25a6c205a2ed None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.690630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 530/1063] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:43.695735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.696015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.696284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.714386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:43.715574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:43.760789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30b7b6-9ccd-467b-a6c8-39ae6a79a713 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.785733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 531/1064] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:43.795703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.796430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.797153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.797153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.797842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:43.798218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:43.799696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:43.808642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 534/1065] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:43.809496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.817999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:43.855160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:43.884222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:43.921418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549b2f7-d96b-4f0b-ad34-585fcf556e44 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.929223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105b16e-3668-48e4-bc3e-7e69320a43fc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:43.959088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.959844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.960245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.960617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.965850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:43.966467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:43.966741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:43.967034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:43.978406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:43.980108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:43.995909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:43.997032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:43.998136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:43.998136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:43.998136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:43.998626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 535/1066] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:44.005135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.035449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.036916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.102717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.112255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.124891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c771-1792-48cb-b771-250d43ad74b9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.153109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 532/1067] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:44:44.158641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.165275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.166325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.167399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.167872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.169464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:44.169998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.173447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.195045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 536/1068] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:44.202424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.263819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57856ac-8b74-49c2-9d74-77c24fca117e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.304545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.305173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.305448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.305722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.314066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ab36b-adc4-4935-9df9-8bd51767cc74 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.316788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.318246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:44.333884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:44.334823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:44.336178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:44.336178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:44.336178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:44.336178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 533/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:44.340740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.356206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.356831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.357116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.357683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.362164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.366642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.368181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:44.380399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:44.381345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:44.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 537/1070] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:44.387117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.392952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.450627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d277be8-e69c-4dea-8a69-095154510f95 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.488589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.489196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.489479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.489751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.490661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:44.490661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.492061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.496461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f89c70-869c-4312-9e9b-b67dcc6a1530 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.505306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 534/1071] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:44.535780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.536413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.536689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.536958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.537730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:44.538036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.539587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.562160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 538/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:44.568016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.653445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.677846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f3519-33e4-4876-a3a0-8442bcbd3a35 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.714211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.715169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.715452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.715722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.727210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.728639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:44.744197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:44.745431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:44.746533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:44.746533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:44.746533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:44.746533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 535/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:44.752592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:44.822058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:44.861306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b790e1-4179-43f1-9d3e-6864759267de None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:44.896178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:44.896779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:44.897056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:44.897324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:44.897986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:44.898288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:44.899686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:44.911626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 539/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:44.916996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.018694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399a013-b92f-4e34-949e-714fc1de5e4f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:45.046732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:45.052782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:45.053406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.053685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.053960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.061132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:45.062824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:45.075196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:45.076241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:45.077677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 536/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:45.082633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.190141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804de-daf7-4e22-986a-0f0645889ccb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:45.212020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:45.224354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:45.224951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.225225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.225488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.226176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:45.226473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:45.227915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f8d2c-3c8a-4d80-9d5a-5c6417c81c73 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:45.239999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 540/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:45.245912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.354244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5a01c-ad91-46ae-bed5-4c72e2171a4c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:45.388276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:45.388916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.389190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.390013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/cffd93c2cc654062a35c70900c6f7ad8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.392265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=a136641c41464f1f969fe6c16dfe3501, role_id=cffd93c2cc654062a35c70900c6f7ad8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:45.393703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48537f5-6ee9-4085-8751-2bd04a142f39 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:45.407571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 537/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:45 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/a136641c41464f1f969fe6c16dfe3501/roles/cffd93c2cc654062a35c70900c6f7ad8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:45.469525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e161-7c6e-44c9-a33d-898aa62516fd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.470043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e161-7c6e-44c9-a33d-898aa62516fd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.470043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9e161-7c6e-44c9-a33d-898aa62516fd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.471397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 541/1078] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:45.477101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.477446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.477726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.497528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:45.499455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:45.550708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:45.574962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 538/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:45.597065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e161-7c6e-44c9-a33d-898aa62516fd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.699781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9e161-7c6e-44c9-a33d-898aa62516fd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:45.733885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:45.734418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.734691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.734949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.744935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:45.750132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:45.751251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:45.752123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 542/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:45.757235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.857492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cf3ca-4740-470b-96f3-513a3f578f05 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:45.890693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:45.891392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:45.891666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:45.891931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:45.892581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:45.892874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:45.894219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:45.902446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 539/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:45.909140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:45.952361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:46.010145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1e32-41c6-4946-bec5-922721daf44c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:46.044337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:46.044953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:46.045229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:46.045497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] PATH_INFO: `/v3/users/819271fd689f4ee2a2212493bee2564d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:46.047331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] RBAC: Authorizing `identity:get_user(user_id=819271fd689f4ee2a2212493bee2564d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:46.048760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:46.050480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 543/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:45 2022] GET /identity/v3/users/819271fd689f4ee2a2212493bee2564d => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:46.056515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:46.101158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:46.160121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697074b-b7d4-4c06-b616-375eca127c9a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:46.196055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:46.196681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:46.196960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:46.197232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:46.198334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:46.199779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e276d-9c02-4fc8-805e-614ea685f4ef None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:46.201401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 540/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:44:46 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:46.207620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:46.306612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4db0a0-b04e-49b0-a2bd-06577033beab None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:46.316544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:46.344560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:46.345797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:46.345797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:46.345797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3/users/819271fd689f4ee2a2212493bee2564d/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:46.347619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1a948696eea46958d179ab44c1b0ea3, user_id=819271fd689f4ee2a2212493bee2564d, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:46.349052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dfab2-a927-4682-a84c-3fdc1fa60293 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:46.362311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 544/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:46 2022] PUT /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3/users/819271fd689f4ee2a2212493bee2564d/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:47.408737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6f79-a15d-4bd0-973f-c64f0d60fa83 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:47.409253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6f79-a15d-4bd0-973f-c64f0d60fa83 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:47.409443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6f79-a15d-4bd0-973f-c64f0d60fa83 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:47.410785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 541/1085] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:47.415949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:47.416327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:47.416497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:47.435460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:47.436684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:47.506134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 545/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:47.528643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d6f79-a15d-4bd0-973f-c64f0d60fa83 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:47.629961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d6f79-a15d-4bd0-973f-c64f0d60fa83 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:47.641879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:47.666771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:47.667357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:47.667633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:47.667912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:47.678443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:47.683031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:47.684095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 17:44:47.685142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:47.687119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 542/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 9 17:44:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:47.690540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:47.786124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:47.789157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656e7d-b517-4e3b-ad8d-bb45112e43d8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:47.824087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:47.824683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:47.824960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:47.825231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:47.825895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:47.826195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:47.828082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:47.836048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 546/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:44:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:47.841631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:47.941083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f0403-71a5-4286-9cc8-bef72461856d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:47.974877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:47.975523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:47.975799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:47.976068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:47.987025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:47.988465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:48.002272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:48.003455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:48.004780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:48.004780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:48.004780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:48.004780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 543/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:48.009357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.112941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c7f3c-0b34-4d68-9650-fc4982a1dcde None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.146526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.147153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.147432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.147702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.148338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:48.148639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.149991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.161451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 547/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:48.167054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.191803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.266527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd1de0-8b7c-447a-ba9c-1e39d69e4558 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.300073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.300709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.300987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.301255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.308861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.310293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6fd7fac3-9e7f-43f7-b4d7-4b0674ddd632 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:44:48.321858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:48.322794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:48.324544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 544/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:48.329211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.352934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0fbd-0823-49e9-8e94-2c6a0e500e1d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.353294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0fbd-0823-49e9-8e94-2c6a0e500e1d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.353489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0fbd-0823-49e9-8e94-2c6a0e500e1d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.353584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.354926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 545/1092] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:44:48.360158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.360439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.360714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.378688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:48.379882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:48.431449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a5a87-facc-48d2-b5de-11a2e10fc723 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.449808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 546/1093] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:44:48.465908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.466513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.466793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.467100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.467857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:48.468161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.469608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.473301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.487680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.488014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 548/1094] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:48.494008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.576518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c5778-a36b-480c-997f-905a765d8b77 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.576894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.596950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c5363-2069-4ea6-b232-2220a6c71261 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.610607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.611329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.611610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.611883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.621759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:44:48.626931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:48.628197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:44:48.629152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 547/1095] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:48.632051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.632708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.632981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.633235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.634111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.702847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.704318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.705776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.733475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d4007-9e96-4866-a5ff-44eac58264f5 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.767765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.768387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.768665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.768956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.769636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:48.769936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.771317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.779670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 548/1096] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:48.785346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.786318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.806468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 549/1097] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:44:48.812686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.886751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d18d0a-7c54-49d9-b95c-14a71600dc81 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.909832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd007b28-0617-482e-b2a0-10d1f34e11ef None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:48.919816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.920433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.921244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.921509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] PATH_INFO: `/v3/users/819271fd689f4ee2a2212493bee2564d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.922618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] RBAC: Authorizing `identity:get_user(user_id=819271fd689f4ee2a2212493bee2564d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.924066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.925610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 549/1098] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/users/819271fd689f4ee2a2212493bee2564d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:48.931146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:48.933778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:48.943801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:48.944414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:48.944687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:48.944947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:48.955931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:48.957308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:44:48.971846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:48.972885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:44:48.973776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:44:48.973776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:44:48.973776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:48.973776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 550/1099] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:44:48.978614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.013946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:49.037150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776633ae-82e0-49b1-a04c-049bfedb614a None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.075575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.076184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.076463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.076795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.085974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ede8-1e89-430f-a34f-db049ecb8680 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.121451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.122053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.122327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.122592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.123328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:49.124157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.125915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.139227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 551/1100] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:49.145410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.161098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.162544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.164396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:49.237467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:49.246495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863495ee-1914-4736-b3d8-835c41311369 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.262754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 550/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:44:49.268515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.284892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.285520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.285752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.286045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.292806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.294245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:44:49.305816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:44:49.306824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:44:49.308108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 552/1102] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:44:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:44:49.313108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.368556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27b0a-3508-4e0c-ad3c-30c575a81838 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.370115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:49.404682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.405325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.405600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.405868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.406996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.408424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.410043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 551/1103] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:44:49 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:49.413455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab8480-10ac-4927-8cf8-822b4a5f6da8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.416013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.448199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.448806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.449069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.449403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.450113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:49.450418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.451945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.463922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 553/1104] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:44:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:49.469831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:49.484834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:49.518123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387853de-06ff-4afc-be30-9e1289466bd2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.551712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.552306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.552587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.552857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.553572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=d1a948696eea46958d179ab44c1b0ea3, user.id=819271fd689f4ee2a2212493bee2564d) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:49.553880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.555329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41c885-ad7c-4333-848a-6ef46b5350f2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.565688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 552/1105] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:49 2022] GET /identity/v3/role_assignments?user.id=819271fd689f4ee2a2212493bee2564d&scope.project.id=d1a948696eea46958d179ab44c1b0ea3&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:49.572444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707134e5-4c5b-4b81-be01-60be76d0e1ad None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:49.612576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:49.613180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:49.613451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:49.613714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:49.614441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] RBAC: Adding query filter params (role.id=cffd93c2cc654062a35c70900c6f7ad8, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=a136641c41464f1f969fe6c16dfe3501) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:49.614743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:49.616211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104bcf-83ea-4160-b438-ac1a4dffb13f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:49.622580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 554/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:49 2022] GET /identity/v3/role_assignments?user.id=a136641c41464f1f969fe6c16dfe3501&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=cffd93c2cc654062a35c70900c6f7ad8 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:44:51.657161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5e4ad-2c49-4f37-914d-01bc49648254 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:51.657752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5e4ad-2c49-4f37-914d-01bc49648254 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:51.657752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5e4ad-2c49-4f37-914d-01bc49648254 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:51.659031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 553/1107] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:51.664279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:51.664728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:51.664844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:51.682398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:51.683610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:51.749341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 555/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:51.773839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5e4ad-2c49-4f37-914d-01bc49648254 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:51.808727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:51.875865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5e4ad-2c49-4f37-914d-01bc49648254 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:51.911116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:51.911692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:51.911973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:51.912244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:51.913357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:51.914792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2f09a-7ac7-4b3c-94a3-fa41fcb3a2c5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:51.916400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 554/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:44:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:51.922275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:51.930578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:52.021910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffee1b7-23c8-46bf-bfbd-59ea570a6fa9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:52.054981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:52.055602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:52.055881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:52.056154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:52.056802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:52.058233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d932-7b38-4635-affa-725117a9ba72 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:52.081487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 556/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:44:51 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:44:54.107194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5df0-7bee-403a-a6de-4e44a2b8896c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.107711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5df0-7bee-403a-a6de-4e44a2b8896c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.107711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5df0-7bee-403a-a6de-4e44a2b8896c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.109593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 555/1111] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:54.116102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d3d66-5acb-4876-8f88-313108427564 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.116519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d3d66-5acb-4876-8f88-313108427564 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.116630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d3d66-5acb-4876-8f88-313108427564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.134134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-956d3d66-5acb-4876-8f88-313108427564 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:54.135339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-956d3d66-5acb-4876-8f88-313108427564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:54.190538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:54.201940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 557/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:54.225077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c5df0-7bee-403a-a6de-4e44a2b8896c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:54.342900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c5df0-7bee-403a-a6de-4e44a2b8896c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:54.344347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:54.378281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:54.378902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.379208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.379481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] PATH_INFO: `/v3/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.380870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] RBAC: Authorizing `identity:get_role(role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:54.382229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:54.383577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 556/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:54 2022] GET /identity/v3/roles/da668845afcd4a95bf6de91627f699b4 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:54.389675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d3d66-5acb-4876-8f88-313108427564 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:54.487258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d3d66-5acb-4876-8f88-313108427564 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:54.520331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:54.520950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.521227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.521496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] PATH_INFO: `/v3/users/c9cba4e8e6ee467588b4229e324c2a98` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.535089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] RBAC: Authorizing `identity:get_user(user_id=c9cba4e8e6ee467588b4229e324c2a98)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:54.536537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:54.538177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 558/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:54 2022] GET /identity/v3/users/c9cba4e8e6ee467588b4229e324c2a98 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:54.543928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:54.644915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45e9d8-6546-4619-86d9-f8fd6017b5bd None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:54.690583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:54.691205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.691481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.691749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.729281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:54.758999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:54.760435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:54.856819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 557/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:44:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:44:54.863382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:54.911968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:54.960733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd2b0b-89a3-43aa-899f-c5f3fa9aee6d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:54.995969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:54.996581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:54.996854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:54.997119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:54.998208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:54.999654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f024-c3ca-41ca-9ebb-bfdce74739c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:55.001249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 559/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:44:54 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:55.007428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:55.075879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:55.107089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331e69c-83be-4a72-8501-9ce8e5d501a7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:55.140620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:55.141251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:55.141463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:55.141724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:55.142428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=d1a948696eea46958d179ab44c1b0ea3, user.id=c9cba4e8e6ee467588b4229e324c2a98) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:44:55.142724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:55.144170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621049f0-91f7-48ba-92c2-e596a84c82fc None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:55.149290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 558/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:44:55 2022] GET /identity/v3/role_assignments?user.id=c9cba4e8e6ee467588b4229e324c2a98&scope.project.id=d1a948696eea46958d179ab44c1b0ea3&role.id=da668845afcd4a95bf6de91627f699b4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:57.102919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e366-0102-486a-8d58-9cb2fd793531 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.103469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e366-0102-486a-8d58-9cb2fd793531 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.103564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04e366-0102-486a-8d58-9cb2fd793531 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.104836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 560/1118] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:57.110218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.110591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.110762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.128968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:57.130128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:57.207715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 559/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:57.229830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e366-0102-486a-8d58-9cb2fd793531 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:57.329999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04e366-0102-486a-8d58-9cb2fd793531 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:57.346388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:57.362807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:57.363449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.363726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.363994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] PATH_INFO: `/v3/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.365356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] RBAC: Authorizing `identity:get_role(role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:57.366700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:57.368037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 561/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:57 2022] GET /identity/v3/roles/da668845afcd4a95bf6de91627f699b4 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:57.374388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:57.433400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:57.471384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9f134-f893-4ab7-8886-f62ba3afa43c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:57.505036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:57.505651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.505926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.506195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] PATH_INFO: `/v3/users/c9cba4e8e6ee467588b4229e324c2a98` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.507433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cba4e8e6ee467588b4229e324c2a98)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:57.508738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:57.510280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 560/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:57 2022] GET /identity/v3/users/c9cba4e8e6ee467588b4229e324c2a98 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:57.516195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:57.612369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fea55-d3f9-42f7-9603-929b7c99b05e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:57.645172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:57.645777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.646052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.646319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.647429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:57.648841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65404c96-725e-4cdf-a6f1-d954b7b93d46 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:57.650442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 562/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:44:57 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:44:57.656524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:57.756782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303733a-d81c-46db-82ef-18c09262d5e6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:44:57.785031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:44:57.793402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:44:57.794080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:57.794341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:57.794612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3/users/c9cba4e8e6ee467588b4229e324c2a98/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:57.797272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1a948696eea46958d179ab44c1b0ea3, user_id=c9cba4e8e6ee467588b4229e324c2a98, role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:44:57.798695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb007e1-cced-4d48-a53f-6500a87f1746 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:44:57.811643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 561/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:44:57 2022] PUT /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3/users/c9cba4e8e6ee467588b4229e324c2a98/roles/da668845afcd4a95bf6de91627f699b4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:44:59.805539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14c0d-c892-4e75-a2de-9b4cada35057 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:59.806106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14c0d-c892-4e75-a2de-9b4cada35057 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:59.806106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e14c0d-c892-4e75-a2de-9b4cada35057 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:59.807446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 563/1124] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:44:59.813393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:44:59.813829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:44:59.813937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:44:59.833222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:44:59.834416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:44:59.903400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 562/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:44:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:44:59.928150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e14c0d-c892-4e75-a2de-9b4cada35057 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:44:59.990394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:00.036891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e14c0d-c892-4e75-a2de-9b4cada35057 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:00.073180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:00.073810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:00.074088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:00.074356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] PATH_INFO: `/v3/roles/da668845afcd4a95bf6de91627f699b4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:00.075760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] RBAC: Authorizing `identity:get_role(role_id=da668845afcd4a95bf6de91627f699b4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:00.077174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:00.078487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 564/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:44:59 2022] GET /identity/v3/roles/da668845afcd4a95bf6de91627f699b4 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:00.084325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:00.181328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a360bd-3ded-4b94-b8ed-ea68ff06e83d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:00.204454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:00.216948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:00.217568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:00.217842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:00.218107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] PATH_INFO: `/v3/users/c9cba4e8e6ee467588b4229e324c2a98` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:00.219227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cba4e8e6ee467588b4229e324c2a98)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:00.220641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:00.222208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 563/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:00 2022] GET /identity/v3/users/c9cba4e8e6ee467588b4229e324c2a98 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:00.228056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:00.329741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5f6a-770b-48e0-ada1-e632b9251714 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:00.352323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:00.364673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:00.365271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:00.365552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:00.365820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:00.437060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:00.438532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:00.534659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 565/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:45:00.540618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:00.639550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6410d-b4fa-4832-95f2-300a539c00a2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:00.672555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:00.673168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:00.673445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:00.673716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] PATH_INFO: `/v3/projects/d1a948696eea46958d179ab44c1b0ea3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:00.674888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] RBAC: Authorizing `identity:get_project(project_id=d1a948696eea46958d179ab44c1b0ea3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:00.676353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed32ba-fabc-451f-8f2e-bdac521284a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:00.677967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 564/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:00 2022] GET /identity/v3/projects/d1a948696eea46958d179ab44c1b0ea3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:00.684318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:00.707839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:00.780226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120c64-ec57-4c15-b2ce-b7e5adb41218 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:00.813511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:00.814230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:00.814342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:00.814615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:00.815362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] RBAC: Adding query filter params (role.id=da668845afcd4a95bf6de91627f699b4, scope.project.id=d1a948696eea46958d179ab44c1b0ea3, user.id=c9cba4e8e6ee467588b4229e324c2a98) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:00.815666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:00.817076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3bc43-4297-45bb-9391-3fe2de2ef99e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:00.822617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 566/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:45:00 2022] GET /identity/v3/role_assignments?user.id=c9cba4e8e6ee467588b4229e324c2a98&scope.project.id=d1a948696eea46958d179ab44c1b0ea3&role.id=da668845afcd4a95bf6de91627f699b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:02.842907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77e04-6d19-4fe2-9f2f-2225ff20ce20 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:02.843504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77e04-6d19-4fe2-9f2f-2225ff20ce20 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:02.843504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e77e04-6d19-4fe2-9f2f-2225ff20ce20 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:02.844824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 565/1131] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:02.849803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:02.850225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:02.850327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:02.868910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:02.870073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:02.935586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 567/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:02.958508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e77e04-6d19-4fe2-9f2f-2225ff20ce20 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:03.017423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:03.059495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e77e04-6d19-4fe2-9f2f-2225ff20ce20 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:03.093892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:03.094416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:03.094690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:03.095032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:03.096126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:03.097551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46144b-241e-4e52-b3a8-2ec782220e67 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:03.099129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 566/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:03.105002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:03.123477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:03.202755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ffe85-fb2d-451b-b193-1845eaf85daa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:03.236433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:03.237141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:03.237288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:03.237561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:03.238202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:03.239767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657dc9a-ea4c-45ae-8944-7d8e3d866323 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:03.265677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 568/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:45:03 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:45:05.308324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75bf22-8575-4f77-9ca8-1110d532cde6 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:05.308888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75bf22-8575-4f77-9ca8-1110d532cde6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:05.308888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75bf22-8575-4f77-9ca8-1110d532cde6 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:05.310173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 567/1135] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:05.316283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:05.316741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:05.316858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:05.335117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:05.336264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:05.405410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:05.411760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 569/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:05.437061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75bf22-8575-4f77-9ca8-1110d532cde6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:05.540760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75bf22-8575-4f77-9ca8-1110d532cde6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:05.566388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:05.575429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:05.576045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:05.576351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:05.576624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:05.577718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:05.579170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de7f3c-4edd-49dc-a42d-b044c83137a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:05.580718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 568/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:45:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:05.586479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:05.685962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857e4c6-8aee-40d1-93a7-fe728301189e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:05.718429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:05.719140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:05.719316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:05.719582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:05.720203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:05.721613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48878fed-1eeb-460f-ad55-0d51c75da21d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:05.742541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 570/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:45:05 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:45:07.813066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7203fc-7c3d-47c4-83ed-66c1db17e6aa None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:07.813066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7203fc-7c3d-47c4-83ed-66c1db17e6aa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:07.813066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7203fc-7c3d-47c4-83ed-66c1db17e6aa None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:07.813066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 569/1139] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:07.818348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:07.818773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:07.818888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:07.836526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:07.837700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:07.890359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:07.908539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 571/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:07.931991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7203fc-7c3d-47c4-83ed-66c1db17e6aa None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:08.033559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7203fc-7c3d-47c4-83ed-66c1db17e6aa None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:08.068379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:08.069007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:08.069270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:08.069545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:08.080659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:08.085230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:08.086471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:08.087699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 570/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:08.093058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:08.195119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654af99f-9bff-47d2-8d5d-69a3b630e793 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:08.229676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:08.230545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:08.230819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:08.231116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:08.231778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:08.232075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:08.233470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:08.241744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 572/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:08.247687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:08.304511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:08.347938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27c6f2-abe6-424d-8ad4-12e603c5a7f2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:08.389984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:08.390605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:08.390879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:08.391177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] PATH_INFO: `/v3/users/2a63d9beb70d469aac1aeab560d3a453` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:08.408217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] RBAC: Authorizing `identity:get_user(user_id=2a63d9beb70d469aac1aeab560d3a453)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:08.409688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:08.411439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 571/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:08 2022] GET /identity/v3/users/2a63d9beb70d469aac1aeab560d3a453 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:08.418112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:08.478960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:08.544054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c8bd3-bc37-4783-b7b7-29ee0b22d80c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:08.578667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:08.579303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:08.579585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:08.579853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:08.652432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:08.653813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:08.698871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:08.755353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 573/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:45:08.761291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:08.865581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d87aad-dc61-4837-b934-ecc0da78e7aa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:08.879936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:08.900650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:08.901275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:08.901554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:08.901825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] PATH_INFO: `/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:08.903016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=4580d6d9dd26443a9dc063f6d3618ce4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:08.904388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9f9dd-7704-4df2-85dc-2df5a0549ae0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:08.906039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 572/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:08 2022] GET /identity/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:08.912299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:09.015731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94ea-a0fc-404d-a8c1-87be9ac47b47 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:09.051159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:09.051763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:09.052043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:09.052315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:09.053043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=4580d6d9dd26443a9dc063f6d3618ce4, user.id=2a63d9beb70d469aac1aeab560d3a453) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:09.053348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:09.054765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451c8a4-625b-4c0c-ae51-973eecaddade None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:09.061578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 574/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:45:08 2022] GET /identity/v3/role_assignments?user.id=2a63d9beb70d469aac1aeab560d3a453&scope.project.id=4580d6d9dd26443a9dc063f6d3618ce4&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:11.087853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276241bf-165a-4c59-83b2-c2785861be74 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.088310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276241bf-165a-4c59-83b2-c2785861be74 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.088425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276241bf-165a-4c59-83b2-c2785861be74 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.089696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 573/1147] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:11.095337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.095731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.095889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.114221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:11.115442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:11.118113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:11.180205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 575/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:11.202743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276241bf-165a-4c59-83b2-c2785861be74 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:11.303044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276241bf-165a-4c59-83b2-c2785861be74 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:11.339493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:11.340032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.340311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.340582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.351055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:11.355871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:11.358218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:11.360245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 574/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:11.363830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:11.457672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea698e9-c870-4128-9208-5e33d62afb2b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:11.490569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:11.491179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.491454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.491719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.492366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:11.492663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:11.494003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:11.501634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 576/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:11.507704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:11.549115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:11.605112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb680a-c9b1-4a32-92fe-cbdc6a62384c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:11.638956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:11.639598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.639873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.640143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] PATH_INFO: `/v3/users/2a63d9beb70d469aac1aeab560d3a453` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.641267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] RBAC: Authorizing `identity:get_user(user_id=2a63d9beb70d469aac1aeab560d3a453)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:11.642682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:11.644292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 575/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:11 2022] GET /identity/v3/users/2a63d9beb70d469aac1aeab560d3a453 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:11.650003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:11.652852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:11.748729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bcd61-0635-48be-8b2c-c624b7e719d5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:11.781782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:11.782386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.782662 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.782928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] PATH_INFO: `/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.784049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] RBAC: Authorizing `identity:get_project(project_id=4580d6d9dd26443a9dc063f6d3618ce4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:11.785459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a258c0f-86fb-4f38-986d-8df6ea250edf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:11.787066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 577/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:11 2022] GET /identity/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:11.793026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:11.889869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc7607-f23b-4572-a93b-04f1237a7a96 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:11.898655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:11.924646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:11.925288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:11.925561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:11.925828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] PATH_INFO: `/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4/users/2a63d9beb70d469aac1aeab560d3a453/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:11.927682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] RBAC: Authorizing `identity:create_grant(project_id=4580d6d9dd26443a9dc063f6d3618ce4, user_id=2a63d9beb70d469aac1aeab560d3a453, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:11.929091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8e582-5a9e-431d-aa98-702e1993ac49 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:11.941985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 576/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:45:11 2022] PUT /identity/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4/users/2a63d9beb70d469aac1aeab560d3a453/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:45:13.907194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ac0b-d292-4f77-b559-cbd01875d98b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:13.907727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ac0b-d292-4f77-b559-cbd01875d98b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:13.907727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ac0b-d292-4f77-b559-cbd01875d98b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:13.909352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 578/1154] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:13.914541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:13.914941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:13.915119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:13.932792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:13.933951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:13.999348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 577/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:14.022842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ac0b-d292-4f77-b559-cbd01875d98b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.092966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:14.125738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0ac0b-d292-4f77-b559-cbd01875d98b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:14.157366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:14.157933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:14.158207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:14.158475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:14.168032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:14.173101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:14.174211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:45:14.175488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 579/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:14.180191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.272734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:14.276610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d92ea1-db9a-48d4-bef4-908a4cb15dc8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:14.311178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:14.311770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:14.312044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:14.312305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:14.312953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:14.313246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:14.314616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:14.322300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 578/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:14.328188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.428485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26856703-2ae8-4897-a89d-c3b17e6cf390 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:14.455231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:14.461715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:14.462333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:14.462610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:14.462873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] PATH_INFO: `/v3/users/2a63d9beb70d469aac1aeab560d3a453` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:14.464003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] RBAC: Authorizing `identity:get_user(user_id=2a63d9beb70d469aac1aeab560d3a453)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:14.465644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:14.467769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 580/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/users/2a63d9beb70d469aac1aeab560d3a453 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:14.473638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.581927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61826b6-cae9-44ea-8c45-5e6181524cbe None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:14.615678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:14.616270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:14.616548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:14.616820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:14.643118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:14.693079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:14.694486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:14.790475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 579/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:45:14.796447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.891864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1303c5-5efc-482d-a04f-62bcf0024382 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:14.924589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:14.925201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:14.925478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:14.925748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] PATH_INFO: `/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:14.926918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] RBAC: Authorizing `identity:get_project(project_id=4580d6d9dd26443a9dc063f6d3618ce4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:14.928363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e408de-15f8-41e8-88c3-829c4187805d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:14.929957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 581/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/projects/4580d6d9dd26443a9dc063f6d3618ce4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:14.935982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:14.992647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:15.032609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf499-f414-4483-98b3-688015d40548 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:15.065245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:15.065845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:15.066123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:15.066394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:15.067219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=4580d6d9dd26443a9dc063f6d3618ce4, user.id=2a63d9beb70d469aac1aeab560d3a453) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:15.067525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:15.068947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fd49a-13ca-444f-8b88-fa176ed1e935 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:15.074610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 580/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:45:14 2022] GET /identity/v3/role_assignments?user.id=2a63d9beb70d469aac1aeab560d3a453&scope.project.id=4580d6d9dd26443a9dc063f6d3618ce4&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:45:17.127298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0db83-0ffb-4c84-90a5-cb966655d1cd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:17.127825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0db83-0ffb-4c84-90a5-cb966655d1cd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:17.127825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0db83-0ffb-4c84-90a5-cb966655d1cd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:17.129169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 582/1162] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:17.134310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:17.134677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:17.134863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:17.152378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:17.153543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:17.218298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 581/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:17.240570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0db83-0ffb-4c84-90a5-cb966655d1cd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:17.305934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:17.342079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0db83-0ffb-4c84-90a5-cb966655d1cd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:17.381038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:17.381637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:17.381918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:17.382191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:17.448304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:17.449768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:17.512126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:17.546936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 583/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:45:17.552920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:17.608471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:17.658986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7592c6-9cc3-4d81-9ee0-be237d1c6564 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:17.691415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:17.692023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:17.692296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:17.692561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:17.698924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:17.700378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8a9b4d74-619b-4b9c-88ba-8180836fb2aa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:17.711255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:17.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:17.713510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 582/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:45:17 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:17.718872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:17.815127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799d45c-6b54-418b-a955-7f2247009249 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:17.848638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:17.849225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:17.849502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:17.849773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:17.850407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:17.850705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:17.852087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17d3e-4ab6-4196-807c-f8bfd0507b03 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:17.859374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 584/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 9 17:45:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:19.852969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95921-9a7b-4f0c-b125-1a5b0a94a2c2 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:19.853502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95921-9a7b-4f0c-b125-1a5b0a94a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:19.853502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95921-9a7b-4f0c-b125-1a5b0a94a2c2 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:19.854835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 583/1167] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:19.860184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e83fc-95e8-4314-aca8-9a9d467201e7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:19.860595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e83fc-95e8-4314-aca8-9a9d467201e7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:19.860734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e83fc-95e8-4314-aca8-9a9d467201e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:19.880676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-868e83fc-95e8-4314-aca8-9a9d467201e7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:19.881861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-868e83fc-95e8-4314-aca8-9a9d467201e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:19.905110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:19.946611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 585/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:19.969423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95921-9a7b-4f0c-b125-1a5b0a94a2c2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:20.069322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95921-9a7b-4f0c-b125-1a5b0a94a2c2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:20.084119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:20.103664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:20.104196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:20.104470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:20.104743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:20.105660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:20.107533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ff72d-a3dc-4b3b-8e46-7b83871b509b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:20.122551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 584/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Tue Aug 9 17:45:19 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:45:22.181784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af080-4659-411a-9426-5ea71797ac8b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:22.182371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af080-4659-411a-9426-5ea71797ac8b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:22.182371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081af080-4659-411a-9426-5ea71797ac8b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:22.183704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 586/1170] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:22.189435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:22.190081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:22.190081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:22.210744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:22.211971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:22.290634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 585/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:45:22.316199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af080-4659-411a-9426-5ea71797ac8b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:22.428700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081af080-4659-411a-9426-5ea71797ac8b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:22.468120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:22.468848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:22.469124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:22.469394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] PATH_INFO: `/v3/domains/a17f959955494164ac58a15c1f7bd2df` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:22.480712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a17f959955494164ac58a15c1f7bd2df)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:22.482668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d77d1-f9cc-4038-b9c8-27ac9351d839 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:22.484283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 587/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Aug 9 17:45:22 2022] GET /identity/v3/domains/a17f959955494164ac58a15c1f7bd2df => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:22.490787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:22.589629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d3ac-511b-4c89-b2f4-1a8b248dd2ee None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:22.592428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:22.624829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:22.625428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:22.625707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:22.625978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:22.626625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:22.628076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a8247-d065-47eb-8b87-1625cc4e9d3b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:22.655169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 586/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:45:22 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:45:24.704497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901d04-c118-4954-981b-c3bd8146457e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:24.705061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901d04-c118-4954-981b-c3bd8146457e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:24.705163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901d04-c118-4954-981b-c3bd8146457e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:24.706456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 588/1174] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:24.711678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:24.712120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:24.712221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:24.730636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:24.731882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:24.801286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 587/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:24.826001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c901d04-c118-4954-981b-c3bd8146457e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:24.848553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:24.926463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c901d04-c118-4954-981b-c3bd8146457e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:24.960544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:24.961090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:24.961363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:24.961631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] PATH_INFO: `/v3/domains/a17f959955494164ac58a15c1f7bd2df` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:24.962717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a17f959955494164ac58a15c1f7bd2df)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:24.964162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a37ef-239a-443d-b6b4-c5a7c052d2a0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:24.965734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 589/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Aug 9 17:45:24 2022] GET /identity/v3/domains/a17f959955494164ac58a15c1f7bd2df => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:24.971528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:25.051350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:25.067003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8acbc44-ce42-4b9e-bbe1-4c052738cf62 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:25.106106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:25.106692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:25.106991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:25.107268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:25.107909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:25.109339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7977a-a1ee-4ff1-8699-2d3de845008c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:25.130404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 588/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:45:24 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:45:27.207009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650877d-e5b0-4ab5-acb6-a9fdd2c312d8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.207542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650877d-e5b0-4ab5-acb6-a9fdd2c312d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.207542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650877d-e5b0-4ab5-acb6-a9fdd2c312d8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.208894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 590/1178] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:27.214405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.214874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.215001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.234261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:27.235483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:27.308207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 589/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:27.331435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650877d-e5b0-4ab5-acb6-a9fdd2c312d8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:27.429478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:27.445342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650877d-e5b0-4ab5-acb6-a9fdd2c312d8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:27.492234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:27.492861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.493141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.493410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.503622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:27.508304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:27.509480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:45:27.510782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 591/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:27.516028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:27.585133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:27.615415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc690c-a6ec-4f3d-94a4-8826c60efc6e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:27.648598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:27.649204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.649481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.649751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.650407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:27.650707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:27.652091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:27.660647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 590/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:27.666498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:27.764015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80bfb4-90b9-4cdc-a14f-f29d1ed70fe4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:27.796905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:27.797515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.797792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.798064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] PATH_INFO: `/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.811494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] RBAC: Authorizing `identity:get_user(user_id=a0e64cbf87ce4b47880131c8e0b3c3ef)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:27.812930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:27.814594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 592/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:27 2022] GET /identity/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:27.820533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:27.919356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8293f2-2855-405d-b224-e5e48ec4ebaa None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:27.953229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:27.953814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:27.954095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:27.954362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:27.956856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:28.022473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:28.023921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:28.120344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 591/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:45:28.127190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:28.135911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:28.225715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2441a0-9f8d-41d8-ab77-2bb6512ac852 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:28.264994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:28.265614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:28.265897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:28.266169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] PATH_INFO: `/v3/projects/4042074ffb094350ac1e27beeba577f3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:28.267308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] RBAC: Authorizing `identity:get_project(project_id=4042074ffb094350ac1e27beeba577f3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:28.268744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83cb9b-2a78-4e76-b243-b8c915569a17 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:28.270346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 593/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:28 2022] GET /identity/v3/projects/4042074ffb094350ac1e27beeba577f3 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:28.276571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:28.308983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:28.377228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc82af-a23c-4501-bc00-e45d87096387 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:28.409761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:28.410341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:28.410623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:28.410890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:28.411639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=4042074ffb094350ac1e27beeba577f3, user.id=a0e64cbf87ce4b47880131c8e0b3c3ef) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:28.411937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:28.413360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cef3f4-9f06-43b0-adbf-3bc6df711733 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:28.418746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 592/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:45:28 2022] GET /identity/v3/role_assignments?user.id=a0e64cbf87ce4b47880131c8e0b3c3ef&scope.project.id=4042074ffb094350ac1e27beeba577f3&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:30.409712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09341-b7f8-4043-b209-85abca0d818e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:30.410301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09341-b7f8-4043-b209-85abca0d818e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:30.410301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc09341-b7f8-4043-b209-85abca0d818e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:30.411600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 594/1186] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:30.417159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:30.417603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:30.417714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:30.436709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:30.437893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:30.504677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 593/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:30.527923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc09341-b7f8-4043-b209-85abca0d818e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:30.607282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:30.632615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc09341-b7f8-4043-b209-85abca0d818e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:30.669435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:30.669968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:30.670242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:30.670508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:30.680545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:30.685153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:30.686304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:45:30.687205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 595/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:30.692524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:30.720143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:30.802007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282ac95-70f3-40b4-bb33-f27cbe011fa7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:30.837380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:30.837987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:30.838261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:30.838531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:30.839219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:30.839541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:30.841522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:30.850026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 594/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:30.856176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:30.960108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247f36-dc88-42da-ad67-2657fe40917b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:30.995808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:30.996428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:30.996707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:30.996979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] PATH_INFO: `/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:30.998641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] RBAC: Authorizing `identity:get_user(user_id=a0e64cbf87ce4b47880131c8e0b3c3ef)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:31.000112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:31.001760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 596/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:30 2022] GET /identity/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:31.007607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:31.099600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:31.109527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52715421-9b7e-491e-8a1b-eb80a8f171fe None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:31.143377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:31.143993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:31.144265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:31.144530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] PATH_INFO: `/v3/projects/4042074ffb094350ac1e27beeba577f3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:31.145656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] RBAC: Authorizing `identity:get_project(project_id=4042074ffb094350ac1e27beeba577f3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:31.147087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8229b-7d2c-4e1c-bc69-b911687bada7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:31.148706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 595/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:31 2022] GET /identity/v3/projects/4042074ffb094350ac1e27beeba577f3 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:31.154711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:31.157570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:31.259782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429e700-7730-47ce-bdee-d410d239a627 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:31.292772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:31.293418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:31.293691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:31.294492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] PATH_INFO: `/v3/projects/4042074ffb094350ac1e27beeba577f3/users/a0e64cbf87ce4b47880131c8e0b3c3ef/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:31.296388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] RBAC: Authorizing `identity:create_grant(project_id=4042074ffb094350ac1e27beeba577f3, user_id=a0e64cbf87ce4b47880131c8e0b3c3ef, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:31.297803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72598fee-742f-443d-973c-0e238d4faa00 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:31.311663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 597/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:45:31 2022] PUT /identity/v3/projects/4042074ffb094350ac1e27beeba577f3/users/a0e64cbf87ce4b47880131c8e0b3c3ef/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:45:33.325425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1786a-4ed8-446e-8a87-b0695447d65d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:33.325934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1786a-4ed8-446e-8a87-b0695447d65d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:33.325934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1786a-4ed8-446e-8a87-b0695447d65d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:33.327754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 596/1193] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:45:33.332705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:33.333145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:33.333247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:33.351940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:45:33.353209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:45:33.411287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:33.423331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 598/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:45:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:45:33.446436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1786a-4ed8-446e-8a87-b0695447d65d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:33.565727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1786a-4ed8-446e-8a87-b0695447d65d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:33.600004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:33.600622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:33.600899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:33.601167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:33.612707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:45:33.617624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:45:33.618750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:45:33.619813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 597/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:45:33.625439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:33.732457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01868e5e-98b6-4cf8-85b1-59268ceabdd6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:33.767146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:33.767752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:33.768033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:33.768304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:33.769057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:33.769373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:33.770737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:33.779772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 599/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:33.786135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:33.805336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:33.886020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9687fe-03dd-4935-b340-94dd9cab02ff None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:33.918890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:33.919540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:33.919822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:33.920097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] PATH_INFO: `/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:33.921199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] RBAC: Authorizing `identity:get_user(user_id=a0e64cbf87ce4b47880131c8e0b3c3ef)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:33.922616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:33.924222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 598/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 9 17:45:33 2022] GET /identity/v3/users/a0e64cbf87ce4b47880131c8e0b3c3ef => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:33.930233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:33.985875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:34.031880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d75a55-a3b3-4c38-85cf-5580020ad8d5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:34.065936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:34.066536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:34.066811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:34.067109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:34.135892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:34.137415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:34.189858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:34.234097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 600/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:45:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:45:34.240030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:34.325033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:34.339580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a36f35-f723-4720-b1e0-9b38076ad5b2 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:34.373675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:34.374285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:34.374565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:34.374828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] PATH_INFO: `/v3/projects/4042074ffb094350ac1e27beeba577f3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:34.375949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] RBAC: Authorizing `identity:get_project(project_id=4042074ffb094350ac1e27beeba577f3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:34.377369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf898ff-bf10-414b-9890-03b079531215 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:34.378993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 599/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 9 17:45:34 2022] GET /identity/v3/projects/4042074ffb094350ac1e27beeba577f3 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:34.385396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:45:34.480611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9c006-f4c7-4b17-bd67-78ca631dac08 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:45:34.513949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:45:34.514538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:45:34.514809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:45:34.515101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:45:34.515815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=4042074ffb094350ac1e27beeba577f3, user.id=a0e64cbf87ce4b47880131c8e0b3c3ef) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:45:34.516117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:45:34.517530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81268a8-04e8-42e9-b6ab-678e5ca1ca29 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:45:34.523143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 601/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:45:34 2022] GET /identity/v3/role_assignments?user.id=a0e64cbf87ce4b47880131c8e0b3c3ef&scope.project.id=4042074ffb094350ac1e27beeba577f3&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:45:44.368960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=23 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:45:44.521012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:29.986902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3193-154e-4435-9a0b-f8d9a421eed4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:29.987507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3193-154e-4435-9a0b-f8d9a421eed4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:29.987507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3193-154e-4435-9a0b-f8d9a421eed4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:29.988812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 600/1201] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:29.994813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:29.995395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:29.995686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:30.017926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:30.019434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:30.121360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 602/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:46:30.146047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d3193-154e-4435-9a0b-f8d9a421eed4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:30.284325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d3193-154e-4435-9a0b-f8d9a421eed4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:30.325559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:30.326250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:30.326551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:30.326855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:30.339965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:30.341732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:30.359647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:30.360905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:30.362036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:30.362036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:30.362036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:30.363820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 601/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:30.370255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:30.509551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95182a34-4ba7-4cbf-ab8b-332ea15c028e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:30.552416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:30.559272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:30.560081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:30.560366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:30.560704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:30.561294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:30.561580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:30.563065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c83140-ebc5-4005-b6ce-d7e9e03c72b6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:30.578714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 603/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:30.584696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:30.658911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:30.732179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23844-aabc-47cf-8c19-4536143f089b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:30.780756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:30.781487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:30.781752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:30.782025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:30.782666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:30.784150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43081f3-c58e-4c5b-b74d-5b6090748f1b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:30.807489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 602/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 9 17:46:30 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:46:33.003703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a010-ff0f-45e8-a6e3-a488833ffc5b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.004250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a010-ff0f-45e8-a6e3-a488833ffc5b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.004250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832a010-ff0f-45e8-a6e3-a488833ffc5b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.005864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 604/1206] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:46:33.011429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.011973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.011973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.034277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:33.035585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:33.135020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 603/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:33.157353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a010-ff0f-45e8-a6e3-a488833ffc5b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:33.288416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832a010-ff0f-45e8-a6e3-a488833ffc5b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:33.328937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:33.329582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.329862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.331789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.345387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:33.350317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:33.351535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:33.352595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 605/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:46:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:33.358268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:33.405890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:33.480384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd0088-5e94-4b36-8c8b-79e939ffbe5d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:33.525704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:33.526904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.527223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.527487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.528214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:33.528608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:33.530485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:33.541512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 604/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:46:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:33.547181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:33.606728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:33.669878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9e71a-aaaa-45d3-86fc-50740b42f38c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:33.712333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:33.713049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.713336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.714208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.725457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:33.727234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:33.743834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:33.745005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:33.745914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:33.745914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:33.745914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:33.745914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 606/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:46:33.751792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:33.895752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:33.899041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6d331-d023-4e70-b06b-39ea426013ec None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:33.940789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:33.941553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:33.942019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:33.942296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:33.943158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:33.943461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:33.945261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:33.962839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 605/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:33.968319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:34.094879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a0ab8-3a3d-4f4b-9dfd-c4642e9f6316 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:34.096893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:34.141520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:34.142334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:34.142784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:34.143094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:34.153205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:34.155269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:34.170160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:34.171594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:34.172601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 607/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:34.177426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:34.297099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68f818-fb55-4c39-9d62-e46cfb87617b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:34.338124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:34.338854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:34.339874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:34.340234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:34.341038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:34.341346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:34.342886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:34.365892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 606/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:46:34.372867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:34.525085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aa755-4b48-4cfb-bfc4-125ae7fe03af None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:34.564612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:34.565238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:34.565546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:34.565837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:34.607332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:34.650875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:34.652789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:34.777475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 608/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:46:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:46:34.778873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:34.799877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:34.899576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181450-8dae-4f9e-9b4c-c690bdfd5579 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:34.945375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:34.945375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:34.945375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:34.945375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:34.963449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:34.965456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:34.984337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:34.985390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:34.986735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:34.986735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:34.986735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:34.990171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 607/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:34.991782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:35.073354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:35.112339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a5e7d-1ec8-4113-9fef-293372dd7c82 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:35.158013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:35.158629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:35.158909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:35.159213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:35.159916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:35.160343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:35.161579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:35.176866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 609/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:35.184790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:35.269399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:35.306603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b72bf2-d797-4eac-b589-a481a50cc826 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:35.347107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:35.347835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:35.348185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:35.348525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:35.356194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:35.357779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:35.371397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:35.372538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:35.373864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:35.375137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 608/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:35.380196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:35.499123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866404-d823-4c84-b27a-147ebad16a16 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:35.522391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:35.536107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:35.536706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:35.536984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:35.537254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:35.537944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:35.538247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:35.539695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4c0e4-7636-466c-af9d-33ccf29a241f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:35.552430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 610/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:35.558366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:35.672274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472becb2-8f3d-43e1-9e2f-389c8ba1ed38 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:35.702549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:35.709594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:35.710220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:35.710498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:35.710768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:35.711615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=5b69ade20aca4860a575feb79522e9ec) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:35.711942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:35.713934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9de222-a345-4614-a227-dd8fdf1b509b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:35.720939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 609/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:46:35 2022] GET /identity/v3/role_assignments?user.id=5b69ade20aca4860a575feb79522e9ec&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:37.764791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea45f-492d-433c-9732-a6060ffb4005 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:37.765375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea45f-492d-433c-9732-a6060ffb4005 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:37.765456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ea45f-492d-433c-9732-a6060ffb4005 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:37.766773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 611/1220] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:37.772006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:37.773342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:37.773342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:37.792555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:37.793750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:37.870567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 610/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:37.893211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ea45f-492d-433c-9732-a6060ffb4005 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.017640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ea45f-492d-433c-9732-a6060ffb4005 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.074271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.074931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.075253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.075565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.086480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:38.092433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:38.093635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:38.094543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 612/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:46:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:38.099652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.222883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d3464-2ec3-4497-96f6-dd1c3a532f40 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.246071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:38.257148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.257752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.258028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.258300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.258956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:38.259285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:38.261203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:38.269547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 611/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:46:38.275360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.321434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:38.382816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38a60-2848-418b-be75-cf6dff043dcb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.418030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.418657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.418939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.419237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.431436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:38.433526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:38.448891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:38.450025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:38.451633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:38.451633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:38.451633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:38.451633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 613/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:38.455900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.565611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594f62c-0e15-494d-b739-93bfed3a0c6d None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.599251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.599848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.600121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.600387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.601015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:38.601311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:38.602691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:38.616951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 612/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:38.623149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.726129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:38.734752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0458ac-b644-4e2a-a167-a9478d2ec666 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.769785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.771142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.771422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.771827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.780467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:38.782019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:38.794250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:38.795482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:38.796730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:38.800191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 614/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:38.801126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:38.854426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:38.910563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1d37-bb5a-459d-b073-601a19a34da3 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:38.945617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:38.946213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:38.946494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:38.946763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:38.947521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:38.948336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:38.949745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:38.967688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 613/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:46:38.974090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:39.083442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1edb6-0cca-4d37-a6ef-34a45c36d4e4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:39.118758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:39.119324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:39.120262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:39.120517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:39.132609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:39.134241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:39.150611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:39.151914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:39.153213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:39.153213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:39.153213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:39.153213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 615/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:39.157894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:39.268047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247337a-39b4-4d6c-9ff4-6cd31b334808 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:39.270683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:39.305584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:39.306184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:39.306464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:39.306734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:39.307942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:39.308246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:39.309599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:39.324047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 614/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:39.330150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:39.341145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:39.443254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79e98-6cc9-46a5-bed2-810019c86e4a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:39.485198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:39.485938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:39.486215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:39.486491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:39.494143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:39.496301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:39.509569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:39.510759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:39.511851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:39.513874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 616/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:46:39.517839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:39.627679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52661d-9841-4808-9759-2116ef3b68a6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:39.669996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:39.670650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:39.671091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:39.671450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:39.672152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:39.672493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:39.674391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923488e-f003-4928-bb2e-6fda24902e4a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:39.688646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 615/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:39.694706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:39.771620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:39.811065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0379b-05fb-46c8-917e-97ef146dc6ed None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:39.848895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:39.849631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:39.849943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:39.850225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/5b69ade20aca4860a575feb79522e9ec/roles/c2028bbfcd744f4e9a4b5c05e650495e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:39.867804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=5b69ade20aca4860a575feb79522e9ec, role_id=c2028bbfcd744f4e9a4b5c05e650495e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:39.869381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ad95c-584e-4ec7-8abd-30e6f9132fcb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:39.883517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 617/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:46:39 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/5b69ade20aca4860a575feb79522e9ec/roles/c2028bbfcd744f4e9a4b5c05e650495e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:46:41.916706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e461-7284-4e13-819e-014852359f47 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:41.917268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e461-7284-4e13-819e-014852359f47 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:41.917268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76e461-7284-4e13-819e-014852359f47 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:41.918552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 616/1233] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:46:41.927450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:41.927450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:41.927450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:41.944188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:41.945389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:42.020327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 618/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 17:46:42.046745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e461-7284-4e13-819e-014852359f47 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:42.080680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:42.165189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76e461-7284-4e13-819e-014852359f47 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:42.207566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:42.208073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:42.208375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:42.208613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:42.219569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:42.224214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:42.225392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:42.226470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 617/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:42.232438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:42.280878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:42.338746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8682e-8d8a-4f1c-9815-f3ae07aad6d3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:42.374649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:42.375463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:42.375854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:42.376137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:42.376941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:42.377242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:42.378764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:42.387312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 619/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:42.393197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:42.493456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:42.510040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ee35-be83-48e8-8b5a-e29b28c6a1ec None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:42.547884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:42.548506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:42.548782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:42.549052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:42.559497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:42.560998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:42.576087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:42.577386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:42.578263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:42.578263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:42.578263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:42.578263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 618/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:42.583310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:42.700889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f9ebb-4ac7-47b8-9c1b-fa18712e93de None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:42.711987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:42.740566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:42.741163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:42.741437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:42.741703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:42.742336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:42.742637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:42.744242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:42.757275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 620/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:46:42.762824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:42.879041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2a0cc-9103-4c9e-a948-3ef8fbf93e59 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:42.914821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:42.915664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:42.915998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:42.916353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:42.920165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:42.925704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:42.927390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:42.940894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:42.942219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:42.943423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 619/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:42.948246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:43.051921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a2c81-8af7-4df2-9f6f-98b13680d065 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:43.089252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:43.089853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:43.090128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:43.090396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:43.091155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:43.091456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:43.092869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:43.111593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 621/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:43.117433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:43.225862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339641d-e675-4294-b68d-7c14d862bb47 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:43.264307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:43.264970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:43.265334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:43.265685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:43.339066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:43.346469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:43.348077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:43.449834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 620/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:46:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 17:46:43.455759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:43.484666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:43.562640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c155ae-6c80-46d4-a78a-98c493b199f4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:43.597300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:43.598011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:43.598295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:43.598604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:43.609000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:43.610437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:43.625592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:43.626755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:43.627836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:43.627836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:43.627836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:43.627836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 622/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:43.632606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:43.749461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:43.755654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68caf90e-f9d9-445f-bc9b-561f769316b7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:43.792085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:43.792798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:43.793130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:43.793511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:43.794290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:43.794657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:43.796255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:43.809110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 621/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:43.814988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:43.901435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:43.925133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae164cb-79e9-4803-bcb6-1483270a9e90 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:43.960492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:43.961192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:43.961382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:43.961653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:43.968992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:43.970644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:43.983070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:43.984137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:43.990148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 623/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:43.990751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:44.105264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8ddb9-ee5d-46bc-9bec-f4c826046eaf None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:44.145023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:44.145671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:44.145996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:44.146268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:44.147004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:44.147349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:44.148886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35998c86-64fd-404d-b7d9-982a6c2686ca None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:44.161940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 622/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:46:44.168639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:44.283185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86680d4f-5e37-44a5-9f57-0192e8f37351 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:44.321082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:44.321785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:44.322092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:44.322367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:44.323222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] RBAC: Adding query filter params (role.id=c2028bbfcd744f4e9a4b5c05e650495e, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=5b69ade20aca4860a575feb79522e9ec) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:44.323536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:44.325108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225fe58-660f-4d43-85ef-15e679f8f98b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:44.331559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 624/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:46:44 2022] GET /identity/v3/role_assignments?user.id=5b69ade20aca4860a575feb79522e9ec&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=c2028bbfcd744f4e9a4b5c05e650495e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:46.506170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620f0d-eb10-49db-891c-2a7bd6913954 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:46.506744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620f0d-eb10-49db-891c-2a7bd6913954 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:46.506886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620f0d-eb10-49db-891c-2a7bd6913954 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:46.508364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 623/1247] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:46.514245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:46.514668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:46.514763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:46.521209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:46.535895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:46.537307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:46.618033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 625/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:46.644778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620f0d-eb10-49db-891c-2a7bd6913954 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:46.671329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:46.769390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620f0d-eb10-49db-891c-2a7bd6913954 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:46.804231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:46.804871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:46.805172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:46.805438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:46.816906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:46.821493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:46.823344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:46.824549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 624/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:46.829444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:46.937997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44243-6140-47f3-9083-c5732e1ec718 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:46.970952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:46.976873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:46.977496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:46.977793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:46.978097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:46.979274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:46.979904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:46.981385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:46.995183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 626/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:46.996763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:47.087294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:47.111853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ffd78-c61b-42d2-84bb-c55420b0e95f None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:47.148699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:47.149368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:47.149660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:47.149939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:47.162423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:47.164281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:47.181378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:47.183141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:47.184210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:47.184210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:47.184210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:47.184210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 625/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:47.188862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:47.305233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759f058-bd8e-4350-8790-46660b470b57 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:47.342768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:47.343364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:47.343634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:47.343897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:47.344535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:47.344833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:47.346183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:47.364415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 627/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:47.365585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:47.475623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f4896-7eb5-4c09-ab8a-89fd2964ddb6 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:47.502501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:47.512905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:47.513683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:47.513983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:47.514314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:47.523070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:47.524783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:47.537287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:47.538569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:47.540058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 626/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:47.547217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:47.608053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:47.663297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e39b57-73bd-45f7-b017-8fd8c2e1d367 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:47.706735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:47.707463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:47.707746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:47.708015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:47.708749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:47.709051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:47.710626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:47.731433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 628/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:47.737619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:47.852197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6362bb-eb7f-4960-a900-57c866ce11b7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:47.888658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:47.889185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:47.889505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:47.889778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:47.954033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:47.968309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:47.970385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:48.085663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 627/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:46:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:46:48.092286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:48.207791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0f3d-bdd1-45cc-9f77-9e5cdbcbddf7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:48.221949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:48.249603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:48.250325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:48.250610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:48.250962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:48.262789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:48.264418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:48.280106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:48.281797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:48.282887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:48.282887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:48.282887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:48.286530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 629/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 17:46:48.287615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:48.382343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:48.396880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db09181-85e0-4d6f-a2a8-6d46e34666f0 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:48.439058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:48.439734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:48.440005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:48.440270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:48.440915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:48.441188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:48.442791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:48.456143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 628/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:48.462186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:48.572227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90e8ac-74c9-4a05-a153-4b2538fe8630 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:48.610009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:48.611090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:48.611450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:48.611752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:48.621131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:48.623124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:48.641296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:48.642640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:48.644245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 630/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:48.649896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:48.785602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a43bb-e77f-4e93-97e6-8399eff8f7f7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:48.821312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:48.821909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:48.822185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:48.822475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:48.823194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:48.823489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:48.824891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c88e8-2e8b-4503-a728-326b85bee20c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:48.839222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 629/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:48.846636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:48.881255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:48.972247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf9ceb-7073-4fbc-859f-5a01fbe04eba None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:49.007974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:49.008886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:49.009169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:49.009444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:49.010288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=5b69ade20aca4860a575feb79522e9ec) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:49.010595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:49.012138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2457b7-0cf7-4c7d-bdd6-96396007f637 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:49.031008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 631/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:46:48 2022] GET /identity/v3/role_assignments?user.id=5b69ade20aca4860a575feb79522e9ec&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:51.020729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec6e3-c338-41bf-9260-849ecf66594f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.021261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec6e3-c338-41bf-9260-849ecf66594f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.021261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec6e3-c338-41bf-9260-849ecf66594f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.022597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 630/1261] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:51.028183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.028631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.028726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.046149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:51.047340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:51.113287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 632/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:51.136207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ec6e3-c338-41bf-9260-849ecf66594f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:51.146851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:51.235144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ec6e3-c338-41bf-9260-849ecf66594f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:51.268561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:51.269168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.269448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.269719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.279659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:51.284175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:51.285257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:51.286260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 631/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:51.291352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:51.309590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:51.387800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34fde2-966e-4ec8-accc-c37a03cb15c1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:51.420876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:51.421583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.421863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.422135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.422786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:51.423110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:51.424461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:51.432211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 633/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:51.437923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:51.502935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:51.542394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c0595-9fc1-48aa-a64c-8e824202ce66 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:51.575318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:51.575926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.576201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.576473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.586141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:51.587676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:51.602103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:51.603234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:51.604154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:51.604154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:51.604154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:51.604154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 632/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:51.609187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:51.700445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:51.719950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff5466-d9b1-406b-9c08-9e3574e858ea None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:51.752019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:51.752612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.752894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.753167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.753804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:51.754109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:51.755512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:51.767512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 634/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:51.773195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:51.869726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7afa69-121e-495e-8bd9-c3c1b6bf5ab7 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:51.888805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:51.902337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:51.902943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:51.903246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:51.903513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:51.911463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:51.912892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:51.923391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:51.924488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:51.925833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 633/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:51.930224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.027346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ebe07-66fa-4b63-8963-518dc1e6638c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.060230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.060833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.061081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.061350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.062069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:52.062370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.063802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.073230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:52.081784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 635/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:52.087431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.195306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e02614-fbfd-46a3-bec7-bb7a1db6f9a1 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.228242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.229411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.229948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.230217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.240290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.241724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:52.255638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:52.256795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:52.257804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:52.257804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:52.257804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:52.257804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 634/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:52.262710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.358269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bd192-8ced-4b0f-bb12-4e60ee71a986 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.390925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.391540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.391819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.392089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.392722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:52.393022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.394376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.405190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 636/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:52.411104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.443617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:52.514694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d09e-b0c0-431a-9ee3-727af887c778 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.546499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.547134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.547412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.547681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.554048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.555511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:52.566638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:52.567733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:52.568839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 635/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:52.573686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.634081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:52.679152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84eec3-f305-4a85-9ed8-3a1587226a7d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.712260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.712916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.713190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.713457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.714136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:52.714433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.715872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc662a6b-be21-4732-b5d9-86961e5a5751 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.727251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 637/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:52.733605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:52.824399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:52.833459 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d9752-a7c4-4704-ba37-9f283a5e51da None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:52.867133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:52.867777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:52.868058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:52.868334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] PATH_INFO: `/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/5b69ade20aca4860a575feb79522e9ec/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:52.870187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] RBAC: Authorizing `identity:create_grant(project_id=a67b49d3c33443589b9d7970888f1e4d, user_id=5b69ade20aca4860a575feb79522e9ec, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:52.871654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1423a-152a-4ff9-80ee-7151e4792a8a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:52.883594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 636/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 9 17:46:52 2022] PUT /identity/v3/projects/a67b49d3c33443589b9d7970888f1e4d/users/5b69ade20aca4860a575feb79522e9ec/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:46:54.849613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0297-e22a-48cc-9756-02e2811e5e1e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:54.850218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0297-e22a-48cc-9756-02e2811e5e1e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:54.850218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0297-e22a-48cc-9756-02e2811e5e1e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:54.851511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 638/1274] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:54.856832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:54.857231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:54.857384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:54.875556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:54.876732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:54.941151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 637/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:54.964013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab0297-e22a-48cc-9756-02e2811e5e1e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.061282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab0297-e22a-48cc-9756-02e2811e5e1e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.078965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:55.095751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.096372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.096652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.096925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.107362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 17:46:55.111868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:55.117402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:55.118549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 639/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 9 17:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:55.120005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.196461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:55.214905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2a971-e2a8-49cb-b614-af9acdbcbea8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.248683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.249254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.249528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.249794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.250439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:55.250732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:55.252108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:55.259781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 638/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:55.265578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.366464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa28f00-5446-4034-ac07-da2553bedcbb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.400210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.400822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.401101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.401380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.411095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:55.412550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:55.427082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:55.428219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:55.429485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:55.429485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:55.429485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:55.429485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 640/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:55.434504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.530491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e07-8418-454e-9887-a51405d68ea4 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.563270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.563856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.564132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.564399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.565028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:55.565325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:55.566671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:55.577891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 639/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:55.583598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.614608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:55.682299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14311ea6-d64c-4617-a2f3-9c80707d9558 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.714448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.715074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.715353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.715622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.722951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:55.724402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 17:46:55.736603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:55.737715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:55.738694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 641/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:55.743514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.747231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:55.840286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de9a6-2480-47f2-b57f-4686ee7c0b8b None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:55.883118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:55.883803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:55.883972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:55.884240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:55.884995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:55.885298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:55.886714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:55.903882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 640/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:55.909642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:55.998462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:56.008242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42e07e-38a6-4c91-811e-6404f549d42d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.041494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.042086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.042364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.042636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.108283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.109728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.200443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:56.211540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 642/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 9 17:46:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 17:46:56.217937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:56.278842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:56.316882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a25a28-aae5-4dbe-b86d-c30d303eac9c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.349863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.350470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.350757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.351028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.361247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.362669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 17:46:56.377429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:56.378552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 17:46:56.379643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 17:46:56.379643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 17:46:56.379643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:56.379643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 641/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:56.384366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:56.480906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1c9-a21b-496e-bcda-8329991b08cb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.514226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.514808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.515120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.515396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.516028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:56.516328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.517686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.528491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 643/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 9 17:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:56.534239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:56.636216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214820cc-5f73-46ed-82f0-03cffe896123 None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.650687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:56.670060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.670668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.670942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.671237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.677593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.679057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 17:46:56.690993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:56.692018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: ERROR keystone.server.flask.application Aug 09 17:46:56.693122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 642/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 9 17:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:56.698493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:56.748415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:56.797547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373b04c-ea01-4d39-8b6f-218813d5162a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.829770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.830366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.830644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.830914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.831626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:56.831930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.833343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb56f9b-0407-4bb6-9a04-f07139ce5764 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.845024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 644/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 9 17:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:56.851149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:56.948070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e481d8-199b-4faf-98a2-6d3a72f42c9e None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:56.981619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:56.982198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:56.982472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:56.982736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:56.983475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] RBAC: Adding query filter params (role.id=7c95125003c7415cb3d20dd52bbf8e1c, scope.project.id=a67b49d3c33443589b9d7970888f1e4d, user.id=5b69ade20aca4860a575feb79522e9ec) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:56.983770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:56.985177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0201400-3f63-4cc9-a4fd-ec53ccec08a5 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:56.990702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 643/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 9 17:46:56 2022] GET /identity/v3/role_assignments?user.id=5b69ade20aca4860a575feb79522e9ec&scope.project.id=a67b49d3c33443589b9d7970888f1e4d&role.id=7c95125003c7415cb3d20dd52bbf8e1c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:58.982924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb62f2-8bdc-48ce-bace-b16946e00946 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:58.983483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb62f2-8bdc-48ce-bace-b16946e00946 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:58.983615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb62f2-8bdc-48ce-bace-b16946e00946 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:58.984851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 645/1288] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:46:58.989972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:58.990379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:58.990519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:59.008143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:46:59.009298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:46:59.038621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:59.076111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 644/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:46:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:46:59.098032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb62f2-8bdc-48ce-bace-b16946e00946 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:59.196573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb62f2-8bdc-48ce-bace-b16946e00946 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:59.207374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:46:59.231036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:59.231645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:59.231926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:59.232198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:59.232771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:59.234180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:46:59.242477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:46:59.243711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:46:59.244818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 646/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:46:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:46:59.249651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:59.345891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab75c29-043e-4daf-9129-570cf8488f85 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:59.379054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:59.379646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:59.379925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:59.380196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:59.380813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] RBAC: Adding query filter params (name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:59.381111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:59.382462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1e2d-6f2f-48fd-9f21-2f403a6a3ead None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:59.387754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 645/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 9 17:46:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:46:59.393525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:46:59.490054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fa03f-15e0-4dae-91a7-a47a2a0b36f2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:46:59.523066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:46:59.523647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:46:59.523921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:46:59.524186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:46:59.524798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:46:59.525092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:46:59.526458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5086-8097-4e27-b111-cca57d0ffb98 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:46:59.530638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 647/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 9 17:46:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:47:01.510622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90713701-58d7-4b9e-9040-4ae17a0b3c32 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:01.511210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90713701-58d7-4b9e-9040-4ae17a0b3c32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:01.511210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90713701-58d7-4b9e-9040-4ae17a0b3c32 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:01.512518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 646/1293] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:47:01.517511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f5b9-b4b6-4048-aa58-b730413e388e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:01.517934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f5b9-b4b6-4048-aa58-b730413e388e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:01.518152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509f5b9-b4b6-4048-aa58-b730413e388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:01.535826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f509f5b9-b4b6-4048-aa58-b730413e388e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:47:01.537002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f509f5b9-b4b6-4048-aa58-b730413e388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:47:01.566896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:01.607406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 648/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:47:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:47:01.629729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90713701-58d7-4b9e-9040-4ae17a0b3c32 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:01.663737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:01.746723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90713701-58d7-4b9e-9040-4ae17a0b3c32 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:01.780430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:01.780961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:01.781241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:01.781512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:01.782152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:01.783558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd7316-4418-4c37-9b89-b945c1e3c9f8 None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:01.797952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 647/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 9 17:47:01 2022] POST /identity/v3/services => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:47:03.790334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8754-676e-4088-a206-6f466a048284 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:03.790875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8754-676e-4088-a206-6f466a048284 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:03.790875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8754-676e-4088-a206-6f466a048284 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:03.792741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 649/1296] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:47:03.797756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:03.798194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:03.798307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:03.816404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:47:03.817574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:47:03.889811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 648/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:47:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:47:03.911870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a8754-676e-4088-a206-6f466a048284 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:04.012429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a8754-676e-4088-a206-6f466a048284 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:04.044683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:04.045228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:04.045508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:04.045780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:04.046352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:04.047744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:47:04.056038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:47:04.057571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:04.058565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 650/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:47:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:47:04.063751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:04.082431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:04.162278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88ce9f-32ab-447f-bf32-a996851f3d21 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:04.204979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:04.205981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:04.206467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:04.206947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:04.208119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:47:04.208654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:04.211254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:04.219572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 649/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 9 17:47:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:47:04.225534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:04.257508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:04.325704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f3dcc-c8df-4d48-977b-69b8db480df0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:04.358628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:04.359262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:04.359545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:04.359817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:04.360538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] RBAC: Adding query filter params (interface=public, service_id=b533332e181d4ebda27fb736a388869a, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:47:04.360844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:04.362258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec200c90-4472-436c-856d-a15689b7e55f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:04.367162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 651/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 9 17:47:04 2022] GET /identity/v3/endpoints?service_id=b533332e181d4ebda27fb736a388869a&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:47:04.373479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:04.452414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:04.470053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c097a7-cb76-4857-ac4c-6f005729d19c None admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:04.504027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:04.504613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:04.504894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:04.505164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:04.505764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:04.507147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f24e0-a8e6-4a86-8ccd-41739e182a4b None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:04.513857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 650/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 9 17:47:04 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:47:06.512531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4703e1-fd67-42cc-a93a-ad488177ebd7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:06.513093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4703e1-fd67-42cc-a93a-ad488177ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:06.513093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4703e1-fd67-42cc-a93a-ad488177ebd7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:06.514401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 652/1302] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:47:06.519410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:06.519854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:06.519947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:06.538857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:47:06.540057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:47:06.604819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 651/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:47:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 17:47:06.627811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4703e1-fd67-42cc-a93a-ad488177ebd7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:06.732510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:06.739498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4703e1-fd67-42cc-a93a-ad488177ebd7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:06.772520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:06.773076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:06.773355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:06.773625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:06.774196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:06.775591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:47:06.783859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:47:06.785035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Aug 09 17:47:06.785894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 653/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 9 17:47:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 17:47:06.791665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:06.866650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:06.890414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c8e4a-e7d2-4de3-90e4-ed3389a9c73e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:06.922962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:06.923580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:06.923856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:06.924122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] PATH_INFO: `/v3/services` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:06.924736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:47:06.925030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:06.926370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0366321-e090-45ad-a92f-20c9b24e6b7a None admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:06.931388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 652/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 9 17:47:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:47:06.937705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:47:07.036654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c98dc1-a2dd-472a-a506-b40cbe8b3fa4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:47:07.069097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:47:07.069815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:47:07.069963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:47:07.070236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:47:07.070888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:47:07.072274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be150a-7b7b-4c3e-b469-aad797f58bad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:47:07.092664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 654/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:47:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:47:16.928495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=30 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:47:17.088461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:10.689116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44810c-5fcb-42dc-86bf-1013a7c95107 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:10.690338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44810c-5fcb-42dc-86bf-1013a7c95107 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:10.690941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44810c-5fcb-42dc-86bf-1013a7c95107 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:10.692917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 653/1307] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:10.699459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e84536-e90a-449a-b5a2-e807575606f1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:10.699934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e84536-e90a-449a-b5a2-e807575606f1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:10.700224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e84536-e90a-449a-b5a2-e807575606f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:10.807224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3e84536-e90a-449a-b5a2-e807575606f1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:10.808794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3e84536-e90a-449a-b5a2-e807575606f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:10.967550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 655/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:11.022156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:11.022624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:11.023148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:11.025902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 654/1309] 10.222.0.52 () {54 vars in 965 bytes} [Tue Aug 9 17:48:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:11.033156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9c00-3e28-4831-91ea-4ee0a0f9e061 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:11.033406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9c00-3e28-4831-91ea-4ee0a0f9e061 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:11.033714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9c00-3e28-4831-91ea-4ee0a0f9e061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:11.150029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-365d9c00-3e28-4831-91ea-4ee0a0f9e061 None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:11.151928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-365d9c00-3e28-4831-91ea-4ee0a0f9e061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:11.180362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:11.269980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 656/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Aug 9 17:48:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:48:11.281117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:11.282919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:11.436361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c5c4d-5418-4997-a2de-98777957d843 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:11.543938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:11.544547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:11.544829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:11.545103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:11.546905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:11.609127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:11.671850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:11.673324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724550-a234-4dde-872e-dffc2f9aea05 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:11.697211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 655/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:14.596953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:14.597472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:14.597714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:14.599670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 657/1312] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:14.605097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc00d1-0d27-43c0-a539-5a4e9b1181f2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:14.605530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc00d1-0d27-43c0-a539-5a4e9b1181f2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:14.605628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc00d1-0d27-43c0-a539-5a4e9b1181f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:14.687335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-21cc00d1-0d27-43c0-a539-5a4e9b1181f2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:14.688548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-21cc00d1-0d27-43c0-a539-5a4e9b1181f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:14.794347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 656/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:14.833513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:14.835188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:14.954399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d3e93-69f8-429d-be30-6fc5f8672b84 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:15.032695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:15.035227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:15.035847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:15.036413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:15.036775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:15.038837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:15.174521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:15.176075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5c19e-49ba-4e7e-a055-a58f33a866b0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:15.198155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 658/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:17.638395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd98e-6954-442b-994a-714a0be67e42 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:17.639148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd98e-6954-442b-994a-714a0be67e42 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:17.639148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0bd98e-6954-442b-994a-714a0be67e42 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:17.640357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 657/1315] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:17.646520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77e7-a2ab-4412-85da-3a6a6b2c7ce8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:17.647026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77e7-a2ab-4412-85da-3a6a6b2c7ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:17.647503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77e7-a2ab-4412-85da-3a6a6b2c7ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:17.729418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d40f77e7-a2ab-4412-85da-3a6a6b2c7ce8 None None] MFA Rules not processed for user `d0ee218e69104872aa22674813a5ef39`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:17.730646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d40f77e7-a2ab-4412-85da-3a6a6b2c7ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:17.781696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:17.834466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 659/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:17.891885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:17.892089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:17.892355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:17.893698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 658/1317] 10.222.0.52 () {54 vars in 965 bytes} [Tue Aug 9 17:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:17.899486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403641-1afc-4335-a838-e8dc5c2fdd6c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:17.899773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403641-1afc-4335-a838-e8dc5c2fdd6c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:17.900048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403641-1afc-4335-a838-e8dc5c2fdd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:17.976176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3d403641-1afc-4335-a838-e8dc5c2fdd6c None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:17.977924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d403641-1afc-4335-a838-e8dc5c2fdd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:18.052923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 660/1318] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Aug 9 17:48:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:18.063259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:18.065901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:18.081939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:18.220456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc1665-37a8-4fea-80da-19759749d15a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:18.295382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:18.296061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:18.296337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:18.296603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:18.298267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:18.361130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:18.408240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:18.409683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8b55e-66c9-48b2-8a9a-9909860f0e42 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:18.427970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 659/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:21.427939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:21.428500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:21.428500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:21.429861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 661/1320] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:21.435158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8784-8384-42ba-a4c7-a157910f62a0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:21.435538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8784-8384-42ba-a4c7-a157910f62a0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:21.435908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf8784-8384-42ba-a4c7-a157910f62a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:21.506648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e4cf8784-8384-42ba-a4c7-a157910f62a0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:21.508387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e4cf8784-8384-42ba-a4c7-a157910f62a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:21.602427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 660/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:21.641708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:21.643310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:21.695070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:21.748124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68def97c-daa5-40f9-b5e9-f9a64ef323cc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:21.819645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:21.820294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:21.821075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:21.821339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:21.822917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:21.925240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:21.939355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:21.940829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcccdf45-aff5-4413-8163-9a73952e014a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:21.961740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 662/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:24.741379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:24.741911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:24.741911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:24.743900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 661/1323] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:24.749397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ab5b-6e1c-464b-9b27-b16ae94bc7df None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:24.749844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ab5b-6e1c-464b-9b27-b16ae94bc7df None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:24.749938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134ab5b-6e1c-464b-9b27-b16ae94bc7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:24.816091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9134ab5b-6e1c-464b-9b27-b16ae94bc7df None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:24.817298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9134ab5b-6e1c-464b-9b27-b16ae94bc7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:24.926000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 663/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:24.963027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:24.964706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:24.986539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:25.076451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2155b0-d6c1-4bda-940b-ab4ad9f63b05 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:25.153658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:25.154259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:25.154538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:25.154809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:25.156426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:25.232142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:25.270402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:25.271828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0c5f8-5836-4a29-82e9-78fe907fb709 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:25.291514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 662/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:27.366368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:27.366930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:27.366930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:27.368319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 664/1326] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:27.373503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b221c-965f-4cc0-aa5d-330d73a3b5e2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:27.373898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b221c-965f-4cc0-aa5d-330d73a3b5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:27.374055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b221c-965f-4cc0-aa5d-330d73a3b5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:27.439999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3c9b221c-965f-4cc0-aa5d-330d73a3b5e2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:27.441193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3c9b221c-965f-4cc0-aa5d-330d73a3b5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:27.539828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 663/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:27.575956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:27.577489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:27.639817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:27.685356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8b16a-be66-4825-b344-52719f3cc930 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:27.763502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:27.764111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:27.764388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:27.764654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:27.766258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:27.873253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:27.875783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:27.877586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1442df-ace5-42d2-8ee5-dade49b989f0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:27.898161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 665/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:29.963691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:29.964254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:29.964254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:29.965541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 664/1329] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:29.970580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47235afa-9d46-45de-b21c-7d56c1dfa8fb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:29.971012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47235afa-9d46-45de-b21c-7d56c1dfa8fb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:29.971145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47235afa-9d46-45de-b21c-7d56c1dfa8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:30.035564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-47235afa-9d46-45de-b21c-7d56c1dfa8fb None None] MFA Rules not processed for user `d0ee218e69104872aa22674813a5ef39`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:30.036764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47235afa-9d46-45de-b21c-7d56c1dfa8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:30.139018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 666/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:30.176797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:30.178322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:30.212076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:30.286473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293d3d9-7a90-4928-8b61-ec24be63081e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:30.362563 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:30.363307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:30.363588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:30.363856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:30.365448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:30.443589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:30.475830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:30.477774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af357-472b-4fa8-af1d-16473013df48 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:30.496059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 665/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:32.787724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:32.788281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:32.792884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:32.794252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 667/1332] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:32.799491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15920d18-a86a-40e5-a36a-0b9644cd175b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:32.799936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15920d18-a86a-40e5-a36a-0b9644cd175b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:32.800036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15920d18-a86a-40e5-a36a-0b9644cd175b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:32.865130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-15920d18-a86a-40e5-a36a-0b9644cd175b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:32.866308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-15920d18-a86a-40e5-a36a-0b9644cd175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:32.959947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 666/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:33.000194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:33.001781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:33.074281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:33.109025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b0c1a-ff24-4a54-a38d-61f58917cb57 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:33.181649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:33.182248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:33.182526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:33.182794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:33.184431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:33.294332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:33.295798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c6c638-5d0b-426d-9fc9-df1ea6c5b452 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:33.306036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:33.315326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 668/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:35.352502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:35.353056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:35.353056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:35.354743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 667/1335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:35.360951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ca3a-ead6-46c0-b54c-6cc1b96be06a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:35.361372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ca3a-ead6-46c0-b54c-6cc1b96be06a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:35.361491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61ca3a-ead6-46c0-b54c-6cc1b96be06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:35.426591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dd61ca3a-ead6-46c0-b54c-6cc1b96be06a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:35.427831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd61ca3a-ead6-46c0-b54c-6cc1b96be06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:35.523145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 669/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:35.563555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:35.565102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:35.604873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:35.668434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8d8f3-c50e-49d2-adc7-7214711e08d0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:35.740947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:35.742139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:35.742417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:35.742681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:35.744259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:35.824331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:35.849397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:35.850817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f9d35-8361-4075-92d9-fedaca2b22f7 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:35.869761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 668/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:38.345671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:38.346208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:38.346208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:38.347620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 670/1338] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:38.353403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24931ed5-5a74-465f-9acd-fef31e40a41b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:38.353806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24931ed5-5a74-465f-9acd-fef31e40a41b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:38.354206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24931ed5-5a74-465f-9acd-fef31e40a41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:38.420167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-24931ed5-5a74-465f-9acd-fef31e40a41b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:38.421367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-24931ed5-5a74-465f-9acd-fef31e40a41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:38.516283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 669/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:38.553591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:38.555176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:38.628796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:38.656447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098fe02-ad43-4a76-81f9-081b0bcc2ae2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:38.729447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:38.730034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:38.730306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:38.730572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:38.732155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:38.836757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:38.838216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09085d96-7fde-44a5-8f20-0216abb05fa3 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:38.856107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 671/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:42.766979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:42.767537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:42.767537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:42.768890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 670/1341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:42.773953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b469-0d86-40f8-86d1-b11f97a363d5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:42.774367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b469-0d86-40f8-86d1-b11f97a363d5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:42.774511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96b469-0d86-40f8-86d1-b11f97a363d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:42.780119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:42.848569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4b96b469-0d86-40f8-86d1-b11f97a363d5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:42.849775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4b96b469-0d86-40f8-86d1-b11f97a363d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:42.947089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 672/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:42.986786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:42.988412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:43.037030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:43.099150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f1a0-d537-42e5-a23e-8976e98aebb2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:43.173327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:43.173924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:43.174201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:43.174469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:43.176159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:43.269409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:43.286814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:43.288303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679055a-3263-4f36-9fcf-194ec4f6623d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:43.306493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 671/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:45.901786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:45.902373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:45.902373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:45.903719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 673/1344] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:45.909513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837088a6-a146-4f58-8b48-e5ccc4ded716 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:45.910043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837088a6-a146-4f58-8b48-e5ccc4ded716 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:45.910131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837088a6-a146-4f58-8b48-e5ccc4ded716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:45.983286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-837088a6-a146-4f58-8b48-e5ccc4ded716 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:45.984529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-837088a6-a146-4f58-8b48-e5ccc4ded716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:46.102553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 672/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:46.139564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:46.141093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:46.229285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:46.246364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c53e0-1f13-4476-bb1e-3ad1b1cedcae None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:46.323047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:46.323573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:46.323866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:46.324131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:46.325734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:46.451311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:46.452745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d081de6-d904-4dd0-89d0-51c8e7cb20d7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:46.471632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 674/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:50.873925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:50.874448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:50.874448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:50.875877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 673/1347] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:50.881379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc8f6-b290-45c2-9baa-09b8d81f8a8a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:50.881862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc8f6-b290-45c2-9baa-09b8d81f8a8a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:50.881985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cc8f6-b290-45c2-9baa-09b8d81f8a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:50.898535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:50.954006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-968cc8f6-b290-45c2-9baa-09b8d81f8a8a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:50.955268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-968cc8f6-b290-45c2-9baa-09b8d81f8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:51.058001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 675/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:48:51.096232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:48:51.097765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:51.168957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:51.215894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047a55e-a018-440b-a998-cdf3a238b6be None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:48:51.290092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:48:51.290683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:51.290953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:51.291249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:51.292853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:51.400887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:48:51.405194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:48:51.406624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d7cac-6103-4d2f-a8dd-43ec26822422 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:48:51.424781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 674/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:48:56.329039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c62a5-69c3-4c1f-a72b-4cd1954b7851 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:56.329560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c62a5-69c3-4c1f-a72b-4cd1954b7851 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:56.329560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c62a5-69c3-4c1f-a72b-4cd1954b7851 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:56.331184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 676/1350] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:48:56.338755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a7d7-d11d-410e-a1cc-a92a239840b2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:48:56.339228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a7d7-d11d-410e-a1cc-a92a239840b2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:48:56.339418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446a7d7-d11d-410e-a1cc-a92a239840b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:48:56.446058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e446a7d7-d11d-410e-a1cc-a92a239840b2 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:48:56.447952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e446a7d7-d11d-410e-a1cc-a92a239840b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:48:56.577178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 675/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:48:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:00.658910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c80d38-c78b-4a6f-bed0-1d1713a74be7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:00.659463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c80d38-c78b-4a6f-bed0-1d1713a74be7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:00.659679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c80d38-c78b-4a6f-bed0-1d1713a74be7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:00.661195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 677/1352] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:00.666859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341e96c-fe45-4fda-9b72-132f81eda185 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:00.667965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341e96c-fe45-4fda-9b72-132f81eda185 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:00.668572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341e96c-fe45-4fda-9b72-132f81eda185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:00.725885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:00.754210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6341e96c-fe45-4fda-9b72-132f81eda185 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:00.755755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6341e96c-fe45-4fda-9b72-132f81eda185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:00.873496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 676/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:49:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:01.056116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=59 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:02.338866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c925f-70d1-4dc5-9d69-77c3b8910e88 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:02.339567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c925f-70d1-4dc5-9d69-77c3b8910e88 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:02.339567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c925f-70d1-4dc5-9d69-77c3b8910e88 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:02.340713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 678/1354] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:02.345911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b385c-57e6-4086-91a6-03f6aa1e7e01 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:02.346348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b385c-57e6-4086-91a6-03f6aa1e7e01 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:02.346460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b385c-57e6-4086-91a6-03f6aa1e7e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:02.421907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-712b385c-57e6-4086-91a6-03f6aa1e7e01 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:02.423145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-712b385c-57e6-4086-91a6-03f6aa1e7e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:02.468498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:02.535998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 677/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:49:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:49:05.905492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0cfc5-0f98-4fac-894d-dcd4b9f28cf3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:05.906501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0cfc5-0f98-4fac-894d-dcd4b9f28cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:05.907452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0cfc5-0f98-4fac-894d-dcd4b9f28cf3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:05.909463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 679/1356] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:05.916036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef679c77-0637-401c-b8fb-c0e1dee5cb9d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:05.916986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef679c77-0637-401c-b8fb-c0e1dee5cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:05.917473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef679c77-0637-401c-b8fb-c0e1dee5cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:06.002849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ef679c77-0637-401c-b8fb-c0e1dee5cb9d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:06.004064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ef679c77-0637-401c-b8fb-c0e1dee5cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:06.121066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 678/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:06.176269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:06.176552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:06.176823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:06.178492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 680/1358] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 9 17:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:06.186193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a37d2-2f89-4270-a668-14a0455807f6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:06.186522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a37d2-2f89-4270-a668-14a0455807f6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:06.186876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a37d2-2f89-4270-a668-14a0455807f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:06.214162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:06.284256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f02a37d2-2f89-4270-a668-14a0455807f6 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:06.285570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f02a37d2-2f89-4270-a668-14a0455807f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:06.419090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 679/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 9 17:49:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:06.425618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:06.427252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:06.585309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372d740-f9f8-45a6-9bd0-206f09927cd1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:06.660907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:06.702750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:06.703440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:06.703815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:06.704188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:06.705900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:06.805139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:06.806884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a2a07-bccb-43ab-a980-28afdc447749 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:06.827344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 681/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:09.102619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:09.103086 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:09.103236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:09.110079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6aa27-4e6d-46cf-8a97-6d3f7e445374 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:09.110504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6aa27-4e6d-46cf-8a97-6d3f7e445374 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:09.110675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6aa27-4e6d-46cf-8a97-6d3f7e445374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:09.115326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 680/1361] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:09.190391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-adf6aa27-4e6d-46cf-8a97-6d3f7e445374 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:09.191618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-adf6aa27-4e6d-46cf-8a97-6d3f7e445374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:09.232572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:09.335689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 682/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:49:09.375503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:09.375503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:09.457197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:09.563116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf95d1b-0830-4f81-8b13-9a4b29a865f1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:09.704348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:09.705220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:09.705782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:09.706261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:09.708178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:09.734232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:09.809729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:09.811493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92bdcd-eed8-4204-a4f6-2f4a29da0eb4 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:09.832037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 681/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:12.196423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb499b-e0a8-4750-bb6f-fac7af14b908 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:12.196974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb499b-e0a8-4750-bb6f-fac7af14b908 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:12.196974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb499b-e0a8-4750-bb6f-fac7af14b908 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:12.198618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 683/1364] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:12.203785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1ab1-b2c2-4497-b51f-7f35deff131c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:12.204295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1ab1-b2c2-4497-b51f-7f35deff131c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:12.204393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f1ab1-b2c2-4497-b51f-7f35deff131c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:12.279877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-132f1ab1-b2c2-4497-b51f-7f35deff131c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:12.281178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-132f1ab1-b2c2-4497-b51f-7f35deff131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:12.382327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:12.390464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 682/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:12.431423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:12.433786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:12.434484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:12.436320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 684/1366] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 9 17:49:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:12.441913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5b17-533a-4c07-b890-d8b03f8dcffa None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:12.442194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5b17-533a-4c07-b890-d8b03f8dcffa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:12.442462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec5b17-533a-4c07-b890-d8b03f8dcffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:12.526029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ccec5b17-533a-4c07-b890-d8b03f8dcffa None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:12.527453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ccec5b17-533a-4c07-b890-d8b03f8dcffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:12.663217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 683/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 9 17:49:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:12.669724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:12.671282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:12.843299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dbaec-2369-44c0-a8c5-aa9e7d5e8f36 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:12.872274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:12.982699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:12.983247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:12.983522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:12.983837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:12.986506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:13.092545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:13.094146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2134acf-b092-4e6d-b7ca-1f8ca42fb687 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:13.119283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 685/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:14.032867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e6c-59af-4e24-b564-3d5d19bd9c90 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:14.033449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e6c-59af-4e24-b564-3d5d19bd9c90 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:14.033449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15e6c-59af-4e24-b564-3d5d19bd9c90 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:14.034883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 684/1369] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:14.039881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eedfc-4756-4421-a6b3-f90ed5d3bf2d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:14.040310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eedfc-4756-4421-a6b3-f90ed5d3bf2d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:14.040512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eedfc-4756-4421-a6b3-f90ed5d3bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:14.081263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:14.117851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8c6eedfc-4756-4421-a6b3-f90ed5d3bf2d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:14.119436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8c6eedfc-4756-4421-a6b3-f90ed5d3bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:14.220405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 686/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:14.572171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:14.572413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:14.572766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:14.574126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 685/1371] 10.222.0.52 () {54 vars in 964 bytes} [Tue Aug 9 17:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:14.578862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c4a9-79f4-441a-bec9-05f518cb04ed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:14.579186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c4a9-79f4-441a-bec9-05f518cb04ed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:14.579912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435c4a9-79f4-441a-bec9-05f518cb04ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:14.662396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8435c4a9-79f4-441a-bec9-05f518cb04ed None None] MFA Rules not processed for user `3098494139cc41359d5c34b2a8428c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:14.664306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8435c4a9-79f4-441a-bec9-05f518cb04ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:14.715901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:14.748867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 687/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Aug 9 17:49:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:14.757630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:14.759147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:14.812543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:14.872905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17956220-39f6-46d6-85b2-43dc328fd623 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:14.952842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:14.953480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:14.953762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:14.954035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:14.955657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:15.064599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:15.078751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:15.081623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415bf3-b015-487d-8e7b-5fc017a9a594 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:15.105458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 686/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:15.204719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:15.205223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:15.205223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:15.206605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 688/1374] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:15.215570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4cac0-f37e-4827-b2b0-2cb4a49fd74e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:15.215570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4cac0-f37e-4827-b2b0-2cb4a49fd74e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:15.215570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4cac0-f37e-4827-b2b0-2cb4a49fd74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:15.284196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-eec4cac0-f37e-4827-b2b0-2cb4a49fd74e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:15.285281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-eec4cac0-f37e-4827-b2b0-2cb4a49fd74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:15.407500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 687/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:15.451224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:15.452783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:15.612404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdc172-e76e-47f9-86ee-36ec2ff3936e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:15.666079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:15.732896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:15.733524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:15.733804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:15.734075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:15.735700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:15.828845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:15.830454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8377d6-4edd-484b-b208-513b85615e1e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:15.850207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 689/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:17.911674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:17.912211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:17.912211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:17.914083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 688/1377] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:17.919664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5c788-5876-41df-b741-bb54cd05127f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:17.920135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5c788-5876-41df-b741-bb54cd05127f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:17.920361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5c788-5876-41df-b741-bb54cd05127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:18.113556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1ba5c788-5876-41df-b741-bb54cd05127f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:18.114762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ba5c788-5876-41df-b741-bb54cd05127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:18.121498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:18.219189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 690/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:18.262913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:18.264494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:18.329457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:18.416804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1304a8-b539-46f4-a513-6169170f3347 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:18.538877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:18.539517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:18.539793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:18.540892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:18.542065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:18.567909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:18.683198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:18.686755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2bdde-4333-469d-a25e-25e7dfba4379 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:18.727206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 689/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:20.818247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:20.818790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:20.818790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:20.820339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 691/1380] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:20.825515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63226f-eca7-4529-ba50-85e2acb945f2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:20.825922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63226f-eca7-4529-ba50-85e2acb945f2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:20.826109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63226f-eca7-4529-ba50-85e2acb945f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:20.897012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1f63226f-eca7-4529-ba50-85e2acb945f2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:20.898281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1f63226f-eca7-4529-ba50-85e2acb945f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:21.000224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:21.008301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 690/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:21.055209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:21.058207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:21.207708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:21.220290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b3866-691f-4e75-8daa-ad8bd2c977bc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:21.342951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:21.343649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:21.344020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:21.344397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:21.346139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:21.461923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:21.463752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055c319-93d4-4bb2-8823-db02b7c7b730 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:21.476225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:21.492629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 692/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:23.595900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:23.596487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:23.596487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:23.597979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 691/1383] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:23.605650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977ce15-b352-412d-b168-7f103659712a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:23.606112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977ce15-b352-412d-b168-7f103659712a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:23.606221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977ce15-b352-412d-b168-7f103659712a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:23.676100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2977ce15-b352-412d-b168-7f103659712a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:23.678034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2977ce15-b352-412d-b168-7f103659712a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:23.818666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 693/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:23.866908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:23.869905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:24.055279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe67df-d067-4227-8e7a-cda02f4b088b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:24.134213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:24.187033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:24.187651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:24.187927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:24.188198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:24.189814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:24.292295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:24.293930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb63c0-ff0e-4645-be84-767f4f98e62e service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:24.316025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 692/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:26.476158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e680d-513f-421c-b044-02a8e023df85 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:26.477673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e680d-513f-421c-b044-02a8e023df85 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:26.477673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e680d-513f-421c-b044-02a8e023df85 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:26.478109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 694/1386] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:26.484107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad2c8-393b-4d1d-990b-e6196ab1f878 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:26.484609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad2c8-393b-4d1d-990b-e6196ab1f878 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:26.484729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad2c8-393b-4d1d-990b-e6196ab1f878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:26.568847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a99ad2c8-393b-4d1d-990b-e6196ab1f878 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:26.571541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a99ad2c8-393b-4d1d-990b-e6196ab1f878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:26.611192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:26.685863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 693/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:26.748732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e680d-513f-421c-b044-02a8e023df85 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:26.751195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-528e680d-513f-421c-b044-02a8e023df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:26.845950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:26.938251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e680d-513f-421c-b044-02a8e023df85 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:27.073688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:27.074297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:27.074573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:27.074843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:27.076481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:27.117787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:27.173319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:27.175010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9d6e8-2b35-44cb-9379-00fb203e8f56 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:27.193751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 695/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:29.422691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:29.423299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:29.423299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:29.424599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 694/1389] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:29.430233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0c82-8025-4dcd-b955-b93a7c5f0f40 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:29.430628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0c82-8025-4dcd-b955-b93a7c5f0f40 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:29.430812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef0c82-8025-4dcd-b955-b93a7c5f0f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:29.510109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8aef0c82-8025-4dcd-b955-b93a7c5f0f40 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:29.511375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8aef0c82-8025-4dcd-b955-b93a7c5f0f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:29.605935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 696/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:29.651251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:29.652880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:29.816536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015be7-b2a6-429c-bfb7-cc3db62de19f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:29.835116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:29.935750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:29.936358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:29.936636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:29.936905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:29.938482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:30.029504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:30.031111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b027de-ff7b-4587-806f-ca36c9fbd1d6 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:30.050133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 695/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:30.305559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362ac3-e21b-495a-8f89-7775a257a1dc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:30.305849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362ac3-e21b-495a-8f89-7775a257a1dc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:30.306121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362ac3-e21b-495a-8f89-7775a257a1dc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:30.307517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 697/1392] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:30.312769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d641-4588-426c-9cbd-e38447f33105 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:30.313056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d641-4588-426c-9cbd-e38447f33105 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:30.313330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07d641-4588-426c-9cbd-e38447f33105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:30.341287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:30.380745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ef07d641-4588-426c-9cbd-e38447f33105 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:30.381925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ef07d641-4588-426c-9cbd-e38447f33105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:30.477843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 696/1393] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:30.533150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:30.533609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:30.533717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:30.535026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 698/1394] 10.222.0.52 () {54 vars in 967 bytes} [Tue Aug 9 17:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:30.540521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b882f6-36d3-4a6a-83df-83bc0c628e62 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:30.540803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b882f6-36d3-4a6a-83df-83bc0c628e62 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:30.541074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b882f6-36d3-4a6a-83df-83bc0c628e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:30.626986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d1b882f6-36d3-4a6a-83df-83bc0c628e62 None None] MFA Rules not processed for user `b46658fdbcde40498e87d37d79609c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:30.628252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d1b882f6-36d3-4a6a-83df-83bc0c628e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:30.671714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:30.711877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 697/1395] 10.222.0.52 () {58 vars in 1064 bytes} [Tue Aug 9 17:49:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:30.719971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:30.721574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:30.740886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:30.842443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cdbda-3d2b-472e-bb10-5f0a36c406db None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:30.917587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:30.918109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:30.918387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:30.918667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:30.920325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:30.989000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:31.031040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:31.032485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94815a5-b8e9-4d35-a0eb-e4eaf4c0e053 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:31.051518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 699/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:31.814112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d89ac-8db2-4db3-92f8-98ec6b42290b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:31.814575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d89ac-8db2-4db3-92f8-98ec6b42290b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:31.814689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d89ac-8db2-4db3-92f8-98ec6b42290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:31.881647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b52d89ac-8db2-4db3-92f8-98ec6b42290b None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:31.883434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b52d89ac-8db2-4db3-92f8-98ec6b42290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:31.973167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 698/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:49:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:32.020071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eba99-df38-46cc-a1f3-d1c03e0f807e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.020586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eba99-df38-46cc-a1f3-d1c03e0f807e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.020671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eba99-df38-46cc-a1f3-d1c03e0f807e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.021977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 700/1398] 10.222.0.52 () {54 vars in 943 bytes} [Tue Aug 9 17:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:32.028656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.028944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.029216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.126580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:32.126862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692556-34d1-4b90-84fc-45f087ec9485 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.127092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692556-34d1-4b90-84fc-45f087ec9485 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.127394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692556-34d1-4b90-84fc-45f087ec9485 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.129059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 701/1399] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:49:32.134680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.135030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.135283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.136328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] MFA Rules not processed for user `eae392d38ade4665b2385bae93b3e688`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:32.137553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:32.209641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:32.211460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:32.247661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 699/1400] 10.222.0.52 () {58 vars in 1040 bytes} [Tue Aug 9 17:49:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:32.259954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:32.269528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0675804-b6e3-445b-b230-e1d20fb13674 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:32.315086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 702/1401] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:49:32.358293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:32.359813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:32.381270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:32.381890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.382195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.383077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.384883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:32.387518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:32.389423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:32.471220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:32.472820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad1da2-2c5c-4e47-9744-acf032dd83c3 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:32.481282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efed4c-1f97-4d3e-b1fc-95001fa76f13 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:32.481447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 700/1402] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:49:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 17:49:32.634126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:32.634812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:32.635188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:32.635517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:32.637291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:32.660792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:32.761820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:32.763518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b559f1a-c675-4c4d-b1d6-3fdd81c2004a service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:32.791517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 703/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:49:34.922557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:34.923192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:34.923192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:34.924639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 701/1404] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:34.930365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914afe8-c2e5-4a29-b50e-e8a22ab08b41 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:34.930789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914afe8-c2e5-4a29-b50e-e8a22ab08b41 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:34.930888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914afe8-c2e5-4a29-b50e-e8a22ab08b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:35.003877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d914afe8-c2e5-4a29-b50e-e8a22ab08b41 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:35.005107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d914afe8-c2e5-4a29-b50e-e8a22ab08b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:35.104640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:35.112084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 704/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:35.156338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:35.157933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:35.322074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:35.325203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19cd68-3c65-4817-a58b-d78855012800 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:35.451379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:35.452168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:35.452442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:35.452709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:35.454294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:35.575704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:35.577219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f465ab-e263-4145-8b31-aafd68e95b6c service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:35.599915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 702/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:37.704836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:37.704836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:37.704836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:37.704836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 705/1407] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:49:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:37.709558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2383f9-d96e-4cb5-b0a3-12cdc58a15cd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:37.709944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2383f9-d96e-4cb5-b0a3-12cdc58a15cd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:37.710228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2383f9-d96e-4cb5-b0a3-12cdc58a15cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:37.722641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:37.845304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7f2383f9-d96e-4cb5-b0a3-12cdc58a15cd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:37.846590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7f2383f9-d96e-4cb5-b0a3-12cdc58a15cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:37.983046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 703/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:38.025552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:38.027131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:38.208713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb949a9-84e4-4126-b03c-2a28a4a8bb81 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:38.290456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:38.345709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:38.346314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:38.346586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:38.346881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:38.348575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:38.453848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:38.455756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec79ef-968c-49a2-b030-21dfcc2a7442 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:38.475785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 706/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:47.978540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=64 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:48.472999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=64 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:54.849806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846ddb8-91c6-42f3-b0b0-0b8df85de2d8 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:54.850612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846ddb8-91c6-42f3-b0b0-0b8df85de2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:54.850612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846ddb8-91c6-42f3-b0b0-0b8df85de2d8 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:54.852667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 704/1410] 10.222.0.52 () {54 vars in 918 bytes} [Tue Aug 9 17:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:54.858994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06acf-4fed-4f64-a9ce-7ac5daef077e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:54.859524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06acf-4fed-4f64-a9ce-7ac5daef077e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:54.859638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b06acf-4fed-4f64-a9ce-7ac5daef077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:54.942635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b7b06acf-4fed-4f64-a9ce-7ac5daef077e None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:54.943886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7b06acf-4fed-4f64-a9ce-7ac5daef077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:55.057405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 707/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Aug 9 17:49:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:55.074561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca8d1-c135-4c38-b0ff-fc77f21fb653 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:55.074846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca8d1-c135-4c38-b0ff-fc77f21fb653 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:55.075226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ca8d1-c135-4c38-b0ff-fc77f21fb653 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:55.076667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 705/1412] 10.222.0.52 () {54 vars in 918 bytes} [Tue Aug 9 17:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:55.082235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8f35-aaa1-451a-a92e-c74c04d61097 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:55.082542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8f35-aaa1-451a-a92e-c74c04d61097 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:55.082819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8f35-aaa1-451a-a92e-c74c04d61097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:55.162038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:55.164499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e2da8f35-aaa1-451a-a92e-c74c04d61097 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:55.166223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e2da8f35-aaa1-451a-a92e-c74c04d61097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:55.280096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 708/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Aug 9 17:49:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:49:57.763532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:57.764083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:57.764083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:57.765402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:57.770921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776103e0-b233-4f3a-bfc5-66b7d768230b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:57.771429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776103e0-b233-4f3a-bfc5-66b7d768230b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:57.771528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776103e0-b233-4f3a-bfc5-66b7d768230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:57.839304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-776103e0-b233-4f3a-bfc5-66b7d768230b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:57.840549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-776103e0-b233-4f3a-bfc5-66b7d768230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:57.900126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:57.950358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:57.994940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:57.996478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:58.151838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafbf62-f439-4112-b86c-9bcc6f6751be None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:58.233424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:58.278187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:58.278789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:58.279093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:58.279364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:58.280968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:58.384210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:49:58.385861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521b197-3a92-4b4d-9c68-1356077203b2 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:49:58.406076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 707/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:49:59.332839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:59.333400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:59.333400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:59.334656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:49:59.340162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29ace9-e872-409c-a2ef-e48668b3cd82 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:59.340559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29ace9-e872-409c-a2ef-e48668b3cd82 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:59.340712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29ace9-e872-409c-a2ef-e48668b3cd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:59.416874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9f29ace9-e872-409c-a2ef-e48668b3cd82 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:49:59.418075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9f29ace9-e872-409c-a2ef-e48668b3cd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:59.444426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:59.515590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:49:59.667337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:49:59.669033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:49:59.819259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb58fd4-fa6b-4bc0-af71-3d6e9aa52949 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:49:59.860814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:49:59.933025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:49:59.933539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:49:59.933812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:49:59.934081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:49:59.935678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:00.022452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:00.024680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c67f1-0f8c-4a33-a49f-d3de407f1a0d service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:00.042604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 711/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:00.926873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:00.927464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:00.927464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:00.928816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:00.933839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa8a1-e703-408e-9cad-edfea8d9e9c9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:00.934287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa8a1-e703-408e-9cad-edfea8d9e9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:00.934393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa8a1-e703-408e-9cad-edfea8d9e9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:00.989188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:01.010999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-21daa8a1-e703-408e-9cad-edfea8d9e9c9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:01.012224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-21daa8a1-e703-408e-9cad-edfea8d9e9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:01.109952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:01.262210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:01.263637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:01.411744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f820472-5027-46e9-98a1-ee8cef2965e0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:01.424058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:01.524906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:01.525434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:01.525715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:01.525987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:01.527594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:01.620297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:01.621885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8a446-d5d7-4ed7-8148-013174a0123f service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:01.640104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 710/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:50:02.500607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:02.501206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:02.501206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:02.502505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:02.507684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b5cf-453a-40ee-ba48-84efbc455d3c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:02.508155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b5cf-453a-40ee-ba48-84efbc455d3c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:02.508273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b5cf-453a-40ee-ba48-84efbc455d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:02.528020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:02.591259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1fc6b5cf-453a-40ee-ba48-84efbc455d3c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:02.592595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1fc6b5cf-453a-40ee-ba48-84efbc455d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:02.693355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:50:02.844296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:02.845921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:02.958570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:02.994772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0dd0c-9bf7-4af4-9050-7af7c7755278 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:03.107127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:03.107779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:03.108321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:03.108586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:03.110153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:03.185859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:03.200355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:03.202007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0826e08-9e0a-4bbc-a8f6-2d32a0d470f6 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:03.225611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 714/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:04.096398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:04.096871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:04.097004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:04.098332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:04.103370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2a72-6a60-4dae-aaa9-3c2263be58f1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:04.103781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2a72-6a60-4dae-aaa9-3c2263be58f1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:04.103925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2a72-6a60-4dae-aaa9-3c2263be58f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:04.178520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c37f2a72-6a60-4dae-aaa9-3c2263be58f1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:04.179756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c37f2a72-6a60-4dae-aaa9-3c2263be58f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:04.278413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:04.429109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:04.431239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:04.513836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:04.579311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c398965-9edd-4c10-8e02-559ef4bcc9f5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:04.693404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:04.693922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:04.694199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:04.694469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:04.696124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:04.744739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:04.786268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:04.787889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254d46-e615-454a-b6bc-40fe2edbe92d service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:04.806326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 713/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:05.610533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:05.611143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:05.611143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:05.612432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:05.617386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c121a-5697-4251-87d7-c9e98b98dc2c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:05.617793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c121a-5697-4251-87d7-c9e98b98dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:05.617944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c121a-5697-4251-87d7-c9e98b98dc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:05.693215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-426c121a-5697-4251-87d7-c9e98b98dc2c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:05.694409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-426c121a-5697-4251-87d7-c9e98b98dc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:05.799378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:05.948553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:05.950101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:05.998433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:06.098791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfc318-cf0d-4a00-8699-13ec0f8934c5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:06.212034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:06.212555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:06.212832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:06.213104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:06.214686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:06.221611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:06.303647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:06.305226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82937431-d51d-40ab-b0a6-32ee8540647d service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:06.323698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 717/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:07.125688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:07.126261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:07.126261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:07.127605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:07.132471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc056a-edd2-46f8-945d-494c5c14fb72 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:07.132896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc056a-edd2-46f8-945d-494c5c14fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:07.133017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc056a-edd2-46f8-945d-494c5c14fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:07.208642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-88bc056a-edd2-46f8-945d-494c5c14fb72 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:07.209898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-88bc056a-edd2-46f8-945d-494c5c14fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:07.276595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:07.305519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:07.454500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:07.456036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:07.473133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:07.604938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ef9e4-0d72-44d8-8db1-52a51587cc11 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:07.694045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:07.720899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:07.721431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:07.721708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:07.721995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:07.723590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:07.814999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:07.816615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416e8c2-ea30-47e1-9522-e6ffe90ac5f6 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:07.834180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 716/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:08.642050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:08.642621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:08.642621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:08.644035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:50:08.649127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0091d-25f7-4ab9-978f-889be492deb7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:08.650125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0091d-25f7-4ab9-978f-889be492deb7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:08.650236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0091d-25f7-4ab9-978f-889be492deb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:08.725937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a9b0091d-25f7-4ab9-978f-889be492deb7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:08.727183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a9b0091d-25f7-4ab9-978f-889be492deb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:08.768886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:08.824153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:08.976786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:08.978236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:09.124214 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa964c87-6aef-492c-ad11-923d17035e65 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:09.177755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:09.244447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:09.244957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:09.245230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:09.245496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:09.247110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:09.335996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:09.337579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a04daa-0588-4781-979e-868cf88e52f2 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:09.355269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 720/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:10.168879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:10.169473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:10.169473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:10.170806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:10.175856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651b58-cfe1-473e-a354-6bc0bbd84451 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:10.176289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651b58-cfe1-473e-a354-6bc0bbd84451 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:10.176412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651b58-cfe1-473e-a354-6bc0bbd84451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:10.254060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7a651b58-cfe1-473e-a354-6bc0bbd84451 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:10.255287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a651b58-cfe1-473e-a354-6bc0bbd84451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:10.257269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:10.350956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:10.504116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:10.505526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:10.650773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e829-75d8-43d4-878a-79fbb52d3394 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:10.677983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:10.886462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:10.887093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:10.887373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:10.887640 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:10.889251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:10.978031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:10.979656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa350c7-be75-4c51-8fb4-82cc317e8a79 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:10.997338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 719/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:50:12.931902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:12.932470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:12.932470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:12.933773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:12.939159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d16-184e-42bd-b77c-f2401ae9a8d0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:12.939560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d16-184e-42bd-b77c-f2401ae9a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:12.939726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa5d16-184e-42bd-b77c-f2401ae9a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:12.980628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:13.006762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0afa5d16-184e-42bd-b77c-f2401ae9a8d0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:13.007981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0afa5d16-184e-42bd-b77c-f2401ae9a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:13.103874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:13.142520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:13.144059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:13.247251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e49805-2256-4512-93b1-b17b32588b39 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:13.278992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:13.319926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:13.320523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:13.320791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:13.321054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:13.323168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:13.429633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:13.431053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9ad0-d5d7-44f9-b96d-d0aa63baa2f9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:13.449604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 723/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:15.619387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:15.619953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:15.619953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:15.621358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:15.626593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a0-9331-4aa1-b582-cffe5fe5dc7d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:15.627012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a0-9331-4aa1-b582-cffe5fe5dc7d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:15.627167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84470a0-9331-4aa1-b582-cffe5fe5dc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:15.697574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a84470a0-9331-4aa1-b582-cffe5fe5dc7d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:15.698788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a84470a0-9331-4aa1-b582-cffe5fe5dc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:15.711589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:15.794927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:15.839208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:15.840800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:15.944694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d367f-3452-4d12-aedc-95474ff285ab None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:15.956934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:16.018228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:16.018965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:16.019338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:16.019609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:16.021404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:16.131137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:16.132565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95767d00-d4bb-4226-bda2-692d1ee6f752 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:16.151153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 722/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:18.503847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:18.504435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:18.504435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:18.505789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:18.511736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a12f02-846a-40e3-b3e8-7109b5174264 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:18.512158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a12f02-846a-40e3-b3e8-7109b5174264 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:18.512370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a12f02-846a-40e3-b3e8-7109b5174264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:18.559351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:18.577687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-94a12f02-846a-40e3-b3e8-7109b5174264 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:18.578867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-94a12f02-846a-40e3-b3e8-7109b5174264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:18.678484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:18.717868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:18.719455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:18.823544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0d298-05a4-483f-b55f-f6a80982333e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:18.864343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:18.898806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:18.899439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:18.899717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:18.899989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:18.901565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:19.013702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:19.015474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496bbd8-b5ae-4eea-8deb-58b67df9421e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:19.033167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 726/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:21.183830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:21.184405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:21.184405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:21.185767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:21.191097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533550d-029e-4441-b442-6cf30804ee28 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:21.191527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533550d-029e-4441-b442-6cf30804ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:21.191653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533550d-029e-4441-b442-6cf30804ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:21.256625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4533550d-029e-4441-b442-6cf30804ee28 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:21.257828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4533550d-029e-4441-b442-6cf30804ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:21.278578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:21.354471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:21.393966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:21.395539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:21.498808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672a39c-6055-4b56-8806-98599c5c5797 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:21.518604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:21.571335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:21.572460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:21.572739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:21.573010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:21.574571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:21.681988 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:21.683379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8a97a-849d-4a0b-b45d-9b3df9693127 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:21.702276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 725/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:23.844083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:23.844675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:23.844675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:23.846342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:23.854330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951121c-5998-4212-9aca-6084e2a8c77a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:23.854885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951121c-5998-4212-9aca-6084e2a8c77a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:23.854885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951121c-5998-4212-9aca-6084e2a8c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:23.932606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:23.935342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2951121c-5998-4212-9aca-6084e2a8c77a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:23.937089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2951121c-5998-4212-9aca-6084e2a8c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:24.041798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:24.083278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:24.091659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:24.201810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1ed11-844e-4b89-8314-c8bb4c2f25fc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:24.255395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:24.281261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:24.281868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:24.282145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:24.282414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:24.284063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:24.406648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:24.409333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00174e37-03b3-43fa-b8d3-ea97f7643d89 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:24.429268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 729/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:26.490109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:26.490654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:26.490654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:26.492014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:26.497888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120148a6-fbc9-46fe-a084-a93450fcf744 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:26.498522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120148a6-fbc9-46fe-a084-a93450fcf744 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:26.498522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120148a6-fbc9-46fe-a084-a93450fcf744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:26.564399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-120148a6-fbc9-46fe-a084-a93450fcf744 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:26.565609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-120148a6-fbc9-46fe-a084-a93450fcf744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:26.593807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:26.661703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:26.701339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:26.702838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:26.805140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6403c7-841c-49c0-aef9-8d4563816f88 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:26.834454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:26.880065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:26.880665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:26.880938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:26.881205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:26.882770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:26.992006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:26.993452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc03d-ad6b-4db4-a916-c55f65ef1b3c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:27.014685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 728/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:29.080527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:29.080975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:29.081073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:29.082382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:29.088088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c82fe-002c-4473-a736-9134e2f1afcd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:29.088532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c82fe-002c-4473-a736-9134e2f1afcd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:29.088630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c82fe-002c-4473-a736-9134e2f1afcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:29.154417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-267c82fe-002c-4473-a736-9134e2f1afcd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:29.155630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-267c82fe-002c-4473-a736-9134e2f1afcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:29.163706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:29.251561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:29.289813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:29.291968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:29.394163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a175c5-3223-43a2-9939-cbbad855ec0e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:29.450138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:29.468679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:29.469277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:29.469573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:29.469828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:29.471448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:29.580720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:29.582149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b52499-1b1e-4d28-af45-d670215f1d93 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:29.600377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 732/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:31.633009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:31.633575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:31.633575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:31.635429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:31.640646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2892-ed61-4a8a-b02c-54384451a3a5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:31.641110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2892-ed61-4a8a-b02c-54384451a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:31.641211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b2892-ed61-4a8a-b02c-54384451a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:31.706425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e40b2892-ed61-4a8a-b02c-54384451a3a5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:31.707667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e40b2892-ed61-4a8a-b02c-54384451a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:31.746002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:31.804559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:50:31.842950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:31.844487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:31.950048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a530-67f2-41e8-9ba7-b71baaa6948b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:31.988213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:32.025407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:32.026013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:32.026290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:32.026561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:32.028159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:32.139865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:32.141311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077123-1375-4bcc-9cad-ea482e03e1c2 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:32.159533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 731/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:34.195290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:34.195846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:34.195846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:34.197168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:34.202356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d5a3b-98d1-414c-8482-b9189e6b79fd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:34.202798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d5a3b-98d1-414c-8482-b9189e6b79fd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:34.202899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d5a3b-98d1-414c-8482-b9189e6b79fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:34.267637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-875d5a3b-98d1-414c-8482-b9189e6b79fd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:34.268836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-875d5a3b-98d1-414c-8482-b9189e6b79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:34.287139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:34.366554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:50:34.406784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:34.409743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:34.517880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb040443-a05c-4774-8422-7fe3424bfe3a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:34.583746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:34.591268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:34.591869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:34.592142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:34.592411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:34.594006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:34.702373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:34.704088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c7333-0eb2-4bb7-8c6f-40d5b9a214cc service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:34.727038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 735/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:36.761736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:36.762333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:36.762333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:36.763667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:36.768837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825a6a-1b5c-4931-b8b4-cf8aecd63641 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:36.769283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825a6a-1b5c-4931-b8b4-cf8aecd63641 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:36.769395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825a6a-1b5c-4931-b8b4-cf8aecd63641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:36.834677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3b825a6a-1b5c-4931-b8b4-cf8aecd63641 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:36.835917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3b825a6a-1b5c-4931-b8b4-cf8aecd63641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:36.884330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:36.933330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:36.969338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:36.970884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:37.075066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faee83-e906-44fc-aa6c-0609f816dd1c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:37.122138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:37.147468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:37.148004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:37.148289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:37.148555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:37.150111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:37.257813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:37.259255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3db17b-ec8d-4d24-94b0-baf393773667 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:37.277223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 734/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:46.930939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=22 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:47.274504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=66 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:56.870363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a07-64b9-455d-b4bd-358efaf14c4c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:56.870930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a07-64b9-455d-b4bd-358efaf14c4c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:56.870930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd688a07-64b9-455d-b4bd-358efaf14c4c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:56.872286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 737/1471] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:56.877325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05901b58-5e86-467f-955c-f65f02118e63 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:56.877729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05901b58-5e86-467f-955c-f65f02118e63 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:56.877867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05901b58-5e86-467f-955c-f65f02118e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:56.942786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-05901b58-5e86-467f-955c-f65f02118e63 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:56.943992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-05901b58-5e86-467f-955c-f65f02118e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:57.041290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 735/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:57.126993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:57.127283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:57.127555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:57.128882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 738/1473] 10.222.0.52 () {54 vars in 967 bytes} [Tue Aug 9 17:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:57.133881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2524f-5903-4faa-a82e-860f01cca1a7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:57.134169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2524f-5903-4faa-a82e-860f01cca1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:57.134442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2524f-5903-4faa-a82e-860f01cca1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:57.210601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f7a2524f-5903-4faa-a82e-860f01cca1a7 None None] MFA Rules not processed for user `b46658fdbcde40498e87d37d79609c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:57.211837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f7a2524f-5903-4faa-a82e-860f01cca1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:57.232741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:57.288402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 736/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Tue Aug 9 17:50:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:57.296646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:57.298156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:57.399833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c390e1-1ae1-4bcf-818b-8334cd954774 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:50:57.472359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:50:57.472954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:57.473223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:57.473487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:57.475100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:57.528801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:50:57.587253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:50:57.589219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8790248-61ab-48ca-a09b-0db367fc0f2b service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:50:57.608212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 739/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:50:59.727798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:59.728378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:59.728378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:59.729687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 737/1476] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:50:59.735027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b146-9ac9-45dd-9416-f187e2809466 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:50:59.735445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b146-9ac9-45dd-9416-f187e2809466 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:50:59.735569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b146-9ac9-45dd-9416-f187e2809466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:50:59.801613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e308b146-9ac9-45dd-9416-f187e2809466 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:50:59.803121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e308b146-9ac9-45dd-9416-f187e2809466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:50:59.899063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 740/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:50:59.921735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:50:59.923292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:00.047418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ce89e-7bad-4c40-a830-d3b82046c6cd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:00.097562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:00.142866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:00.143496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:00.143773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:00.144043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:00.144668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:00.146397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:51:00.146397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:51:00.146397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:51:00.147578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7941f75-ddcd-49b6-87cf-cbb7dd743da0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:00.165840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 738/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 9 17:50:59 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:51:02.187405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:02.187936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:02.187936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:02.189298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 741/1479] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:51:02.194729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3ae8-b093-446d-a90d-04fb1a8358e1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:02.195211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3ae8-b093-446d-a90d-04fb1a8358e1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:02.195310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd3ae8-b093-446d-a90d-04fb1a8358e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:02.262117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4afd3ae8-b093-446d-a90d-04fb1a8358e1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:51:02.263364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4afd3ae8-b093-446d-a90d-04fb1a8358e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:02.359312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 739/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:51:02.402555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:51:02.404141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:02.413761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:02.549592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7904a1-e11c-4782-9188-1b7b65717f22 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:02.630579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:02.664735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060a504-997e-4869-975a-627b7a846055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:02.665336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a504-997e-4869-975a-627b7a846055 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:02.665613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a504-997e-4869-975a-627b7a846055 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:02.665884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a504-997e-4869-975a-627b7a846055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:02.667567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6060a504-997e-4869-975a-627b7a846055 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:02.756292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060a504-997e-4869-975a-627b7a846055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:02.757877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060a504-997e-4869-975a-627b7a846055 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:02.776282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 742/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:51:04.945631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:04.946165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:04.946165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:04.947456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 740/1482] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:51:04.952677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7eeb-075b-4f74-8f7b-14af1cf92738 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:04.953129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7eeb-075b-4f74-8f7b-14af1cf92738 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:04.953244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7eeb-075b-4f74-8f7b-14af1cf92738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:05.019618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-08ce7eeb-075b-4f74-8f7b-14af1cf92738 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:51:05.020825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-08ce7eeb-075b-4f74-8f7b-14af1cf92738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:05.069376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:05.119449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 743/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:51:05.155551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:51:05.157109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:05.181388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:05.263175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4ea4e-99ee-4eae-909e-678f570512bf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:05.339688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:05.340293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:05.340573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:05.340840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:05.342443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:05.419057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:05.454101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:05.455575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e0e0-1c86-4f65-a311-0fd98e2b5485 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:05.474050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 741/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:51:07.570820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:07.571473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:07.571473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:07.572778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 744/1485] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:51:07.578571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392b48-375c-4fc8-8d85-be629a42d2fe None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:07.579070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392b48-375c-4fc8-8d85-be629a42d2fe None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:07.579165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43392b48-375c-4fc8-8d85-be629a42d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:07.648321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-43392b48-375c-4fc8-8d85-be629a42d2fe None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:51:07.649532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-43392b48-375c-4fc8-8d85-be629a42d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:07.746345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 742/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:51:07.784807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:51:07.786326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:07.891752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76693351-f731-46a3-8019-fa5833bcabbf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:07.964861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:07.966944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:07.967809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:07.968215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:07.968654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:07.970259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:08.077619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:08.079106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6cefb-3a1c-4683-8635-56ec4fd4b2c4 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:08.098029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 745/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:51:10.216599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:10.217129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:10.217129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:10.218424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 743/1488] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:51:10.223904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd7dba-5890-4793-956f-3e605bd869fd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:10.224344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd7dba-5890-4793-956f-3e605bd869fd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:10.224449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd7dba-5890-4793-956f-3e605bd869fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:10.289467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dafd7dba-5890-4793-956f-3e605bd869fd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:51:10.290671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dafd7dba-5890-4793-956f-3e605bd869fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:10.345178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:10.389727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 746/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:51:10.429078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:51:10.430584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:10.467833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:10.534839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee983ab-d526-4743-a6ff-3810ba5e80fa None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:10.607504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:10.608100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:10.608382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:10.608650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:10.610226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:10.697119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:10.725962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:10.727463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0876dfd-dbb7-402d-a873-ebffe30cae9d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:10.745875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 744/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:51:12.957172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:12.957725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:12.957725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:12.959109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 747/1491] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 9 17:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:51:12.964189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2336-7686-4545-8ca8-3ed84a2c31c8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:12.964640 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2336-7686-4545-8ca8-3ed84a2c31c8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:12.964749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2336-7686-4545-8ca8-3ed84a2c31c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:13.031279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-056c2336-7686-4545-8ca8-3ed84a2c31c8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:51:13.032502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-056c2336-7686-4545-8ca8-3ed84a2c31c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:13.140430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 745/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 9 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:51:13.179701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:51:13.181223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:13.283887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58c1e7-147f-4c91-a7c7-f640f7aaa12e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:51:13.355750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:51:13.356882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:51:13.357160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:51:13.357430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:51:13.359065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:51:13.368428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:13.469522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:51:13.470878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b7f79-c5ab-4a19-a444-de6d189e850c service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:51:13.488779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 748/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:51:23.138536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=82 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:51:23.486241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=50 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:25.115199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:25.116000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:25.116105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:25.193098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:52:25.194295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:25.290084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 746/1494] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:52:25.299361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:25.299866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:25.299971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:25.375835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:52:25.377067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:25.472740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 749/1495] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:52:25.477506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:25.478722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:25.490926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7ab40-d679-4e16-9886-113660b3da42 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:25.550479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:25.584016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:25.584609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:25.584884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:25.585148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:25.585786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:52:25.586068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:25.587116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:25.587116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:52:25.587116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:52:25.587993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:25.600163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 747/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:52:25.608487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:25.617808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a67ad-6e4c-4669-8941-e5d0657b981b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:25.680298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:25.714807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:25.715425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:25.715696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:25.715960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:25.716578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:25.718865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:25.718865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:52:25.718865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:52:25.720079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:25.773770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 750/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:52:25 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:52:25.777923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:25.780056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:25.812080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3abcd-b794-41c8-a19c-0a5e235e32fe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:25.906177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:25.906787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:25.907114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:25.907368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:25.908000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:25.909011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:25.909011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:52:25.909011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:52:25.910274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:25.953106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 748/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:52:25 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:52:25.956758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:25.958262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.020156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc95c-08d5-4d50-8878-a654e2d5ae4d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:26.105904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:26.120577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:26.121190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.121469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.121744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.122384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:26.123917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:26.123917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:52:26.123917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:52:26.124804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:26.137046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 751/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:52:26.142059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:26.143585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.154870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1b3fb-f499-40a3-8e5e-7c8ab57e6e44 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:26.179328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:26.250639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:26.251164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.251442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.251709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.252336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:26.253363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:26.253363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:52:26.253363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:52:26.254248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494995d-ce8a-4026-b56e-629e0f4b4c74 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:26.266936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 749/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:52:26.271211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:26.272766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.305999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c81c-6558-4b9a-bd79-9b0cb58c38f3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:26.398655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:26.399330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.399606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.399874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] PATH_INFO: `/v3/projects/f84e286add594ed69f5ffbf155289164/users/dd33cc673eea49ecbbc1cccfc506abeb/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.422202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84e286add594ed69f5ffbf155289164, user_id=dd33cc673eea49ecbbc1cccfc506abeb, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:26.423252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:26.423252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:52:26.423252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:52:26.425211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760ab79-6089-47a3-aeca-f7cb61befffb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:26.450462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 752/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:52:26 2022] PUT /identity/v3/projects/f84e286add594ed69f5ffbf155289164/users/dd33cc673eea49ecbbc1cccfc506abeb/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:52:26.462593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdbc5e-28c8-4687-94d2-413f376970a6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.462885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdbc5e-28c8-4687-94d2-413f376970a6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.463192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdbc5e-28c8-4687-94d2-413f376970a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.521435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c2bdbc5e-28c8-4687-94d2-413f376970a6 None None] MFA Rules not processed for user `dd33cc673eea49ecbbc1cccfc506abeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:52:26.522555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c2bdbc5e-28c8-4687-94d2-413f376970a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.600624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 750/1502] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:52:26.614317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.614604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.614877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.616483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 753/1503] 10.222.0.52 () {54 vars in 964 bytes} [Tue Aug 9 17:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:52:26.621191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd860e-9717-4b24-9afd-368fc8a2c6fc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.621481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd860e-9717-4b24-9afd-368fc8a2c6fc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.621787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd860e-9717-4b24-9afd-368fc8a2c6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.652597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:26.701226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c0dd860e-9717-4b24-9afd-368fc8a2c6fc None None] MFA Rules not processed for user `3098494139cc41359d5c34b2a8428c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:52:26.702433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c0dd860e-9717-4b24-9afd-368fc8a2c6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.776093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 751/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Aug 9 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:52:26.784357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:26.785862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:26.836305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:26.890638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a7e09-9900-41f0-bc20-f75b03c5b38e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:26.964802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:26.965411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:26.965691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:26.965965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:26.968192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:27.069420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:27.070822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:27.074354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:27.089389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 754/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:52:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:52:27.134884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155c928-d243-4e13-a22a-b8b6fd4ba5c6 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:27.135208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155c928-d243-4e13-a22a-b8b6fd4ba5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:27.135479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155c928-d243-4e13-a22a-b8b6fd4ba5c6 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:27.136716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 752/1506] 10.222.0.52 () {52 vars in 830 bytes} [Tue Aug 9 17:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:52:27.141535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:27.142820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:27.241649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51b1b-7c6d-4c32-8fb0-f56636bc6f20 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:27.333487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:27.334117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:27.334291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:27.334555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] PATH_INFO: `/v3/users/dd33cc673eea49ecbbc1cccfc506abeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:27.347189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd33cc673eea49ecbbc1cccfc506abeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:27.348204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:27.348204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:52:27.348204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:52:27.349412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:27.359005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:27.414922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd33cc673eea49ecbbc1cccfc506abeb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:27.423942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd33cc673eea49ecbbc1cccfc506abeb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:27.424403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd33cc673eea49ecbbc1cccfc506abeb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:27.447420 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd33cc673eea49ecbbc1cccfc506abeb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:27.461795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd33cc673eea49ecbbc1cccfc506abeb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:27.467443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 755/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:52:27 2022] DELETE /identity/v3/users/dd33cc673eea49ecbbc1cccfc506abeb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:52:27.471965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:27.472484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:27.472610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:27.527972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:52:27.529236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:27.611001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:27.624775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 753/1508] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:52:27.635172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:27.636388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:27.739025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd24162-7ec0-485a-8602-c38e0cc76557 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:27.798646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:27.819498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:27.820113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:27.820393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:27.820668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:27.822273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:27.939141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:27.940491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af6704-d3a6-40f6-851b-76a9c95223ba service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:27.962666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 756/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:52:28.392024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:52:28.393238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:52:28.405500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b274b71-8f18-4e78-81e4-e62ac59996a0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:52:28.508761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:52:28.509406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:52:28.510555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:52:28.510844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] PATH_INFO: `/v3/projects/f84e286add594ed69f5ffbf155289164` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:52:28.521241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84e286add594ed69f5ffbf155289164)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:52:28.522261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:52:28.522261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:52:28.522261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:52:28.523496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:52:28.581499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84e286add594ed69f5ffbf155289164', 'request_id': 'req-9bf9e930-736e-43e6-8b28-910b12a914f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:28.587417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84e286add594ed69f5ffbf155289164', 'request_id': 'req-9bf9e930-736e-43e6-8b28-910b12a914f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:28.600316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84e286add594ed69f5ffbf155289164', 'request_id': 'req-9bf9e930-736e-43e6-8b28-910b12a914f9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:28.628496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] The token cache is being invalidate because project f84e286add594ed69f5ffbf155289164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:52:28.628773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:28.629202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-9bf9e930-736e-43e6-8b28-910b12a914f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:52:28.630603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 754/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:52:28 2022] DELETE /identity/v3/projects/f84e286add594ed69f5ffbf155289164 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:52:37.959691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=62 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:52:38.624795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:54:00.138395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:00.139031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:00.139031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:00.140300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:00.145005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d92-c843-47bf-aa0b-deea04bd6992 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:00.145455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d92-c843-47bf-aa0b-deea04bd6992 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:00.145565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7d92-c843-47bf-aa0b-deea04bd6992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:00.224919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e61f7d92-c843-47bf-aa0b-deea04bd6992 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:00.226642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e61f7d92-c843-47bf-aa0b-deea04bd6992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:00.328196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:00.482550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:00.484739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:00.604283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24cb9f-947e-4ced-a0d7-d36f9ebd3fed None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:00.609743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:00.610340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:00.610634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:00.610884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:00.619221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:00.620792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba795530-db7c-4e2c-a40b-0deec1ba6dd1 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:00.629969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 758/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:01.461219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f374-c199-477b-b873-285dee033fbf None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:01.461822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f374-c199-477b-b873-285dee033fbf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:01.461822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f374-c199-477b-b873-285dee033fbf None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:01.463134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:01.468082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73dfd-9152-44dd-a112-49c9e86da7b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:01.468496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73dfd-9152-44dd-a112-49c9e86da7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:01.468709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73dfd-9152-44dd-a112-49c9e86da7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:01.487908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f5a73dfd-9152-44dd-a112-49c9e86da7b0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:01.489109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f5a73dfd-9152-44dd-a112-49c9e86da7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:01.497156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:01.651607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0f374-c199-477b-b873-285dee033fbf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:01.665703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0f374-c199-477b-b873-285dee033fbf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:01.671032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:01.671693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:01.671974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:01.672245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:01.680281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:01.681921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a688213-efa0-49b9-9abb-52233e4fd8f9 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:01.693997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 757/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:04.698650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba40985-1c2b-420b-afb2-8b7dee935e2b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:04.699250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba40985-1c2b-420b-afb2-8b7dee935e2b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:04.699250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba40985-1c2b-420b-afb2-8b7dee935e2b None None] PATH_INFO: `/v3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:04.700512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 760/1517] 10.222.0.40 () {54 vars in 925 bytes} [Tue Aug 9 17:54:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:54:04.711412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498154-4362-4747-9a04-83588842a242 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:04.711876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498154-4362-4747-9a04-83588842a242 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:04.711986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2498154-4362-4747-9a04-83588842a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:04.770159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a2498154-4362-4747-9a04-83588842a242 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:04.771481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a2498154-4362-4747-9a04-83588842a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:04.787430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:54:04.840325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 758/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Tue Aug 9 17:54:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:54:09.174252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524c93c-ba7d-46eb-bdd8-139739bb90aa None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:09.174705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524c93c-ba7d-46eb-bdd8-139739bb90aa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:09.174814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524c93c-ba7d-46eb-bdd8-139739bb90aa None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:09.176190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 761/1519] 10.222.0.52 () {54 vars in 961 bytes} [Tue Aug 9 17:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:09.181305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171aa4-0ba9-498c-82c0-355df3c9434b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:09.181727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171aa4-0ba9-498c-82c0-355df3c9434b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:09.181920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171aa4-0ba9-498c-82c0-355df3c9434b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:09.203198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9d171aa4-0ba9-498c-82c0-355df3c9434b None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:09.204375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9d171aa4-0ba9-498c-82c0-355df3c9434b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:09.213574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 759/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Aug 9 17:54:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:09.221350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524c93c-ba7d-46eb-bdd8-139739bb90aa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:09.237384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524c93c-ba7d-46eb-bdd8-139739bb90aa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:09.243214 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:09.244084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:09.244347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:09.244622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:09.253299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:09.254844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1489fa-e5f4-4699-9d79-e1d20b41f444 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:09.265160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 762/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:12.919696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7013c-7072-46c7-b5df-65691e6e37c4 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:12.920297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7013c-7072-46c7-b5df-65691e6e37c4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:12.920297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7013c-7072-46c7-b5df-65691e6e37c4 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:12.921557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:12.926375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a0c5-3a3c-45e6-bee6-65cfd55196f6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:12.926798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a0c5-3a3c-45e6-bee6-65cfd55196f6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:12.926925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a0c5-3a3c-45e6-bee6-65cfd55196f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:12.950279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61d7a0c5-3a3c-45e6-bee6-65cfd55196f6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:12.950279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61d7a0c5-3a3c-45e6-bee6-65cfd55196f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:12.957434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:13.112653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7013c-7072-46c7-b5df-65691e6e37c4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:13.128545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7013c-7072-46c7-b5df-65691e6e37c4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:13.136589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:13.137184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:13.137456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:13.137722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:13.146345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:13.147967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f173216-208e-4544-a18f-10b8b200864d service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:13.157676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 761/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:13.990751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e408d6e-63d0-4910-905b-8762a484ee3e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:13.991362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e408d6e-63d0-4910-905b-8762a484ee3e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:13.991362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e408d6e-63d0-4910-905b-8762a484ee3e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:13.993457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 17:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:13.998728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7109-897a-461f-904c-cf6694f005e7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:13.999257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7109-897a-461f-904c-cf6694f005e7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:13.999362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf7109-897a-461f-904c-cf6694f005e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:14.018386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-05cf7109-897a-461f-904c-cf6694f005e7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:14.019718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-05cf7109-897a-461f-904c-cf6694f005e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:14.028206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:14.179504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e408d6e-63d0-4910-905b-8762a484ee3e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:14.193765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e408d6e-63d0-4910-905b-8762a484ee3e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:14.199679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:14.200259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:14.200535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:14.200830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:14.208738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:14.210892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abe3a8-589f-4ff4-8050-98fd671d64ef service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:14.220058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 765/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:18.335193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:18.335791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:18.335791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:18.337009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:18.342124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde47f-1e4b-4d0f-ae90-4033b7d2894e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:18.342595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde47f-1e4b-4d0f-ae90-4033b7d2894e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:18.342716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bde47f-1e4b-4d0f-ae90-4033b7d2894e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:18.362227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a8bde47f-1e4b-4d0f-ae90-4033b7d2894e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:18.363455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a8bde47f-1e4b-4d0f-ae90-4033b7d2894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:18.372614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:18.450314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:18.452074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:18.541652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5efcfc-597d-4932-ad26-4b9c882b7e3a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:18.549938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:18.550546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:18.550824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:18.551138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:18.561108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:18.562580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4228270-3d2a-439b-9120-bfbf6545b3bf service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:18.573162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 764/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:20.584456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1d015-73d4-4b7c-85d0-c49e75ea582b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:20.585028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1d015-73d4-4b7c-85d0-c49e75ea582b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:20.585028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1d015-73d4-4b7c-85d0-c49e75ea582b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:20.586823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:20.591855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449a4e2-c7cf-4281-9b31-a076b69245e9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:20.592268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449a4e2-c7cf-4281-9b31-a076b69245e9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:20.592409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449a4e2-c7cf-4281-9b31-a076b69245e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:20.611883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5449a4e2-c7cf-4281-9b31-a076b69245e9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:20.613062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5449a4e2-c7cf-4281-9b31-a076b69245e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:20.621041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:20.656223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1d015-73d4-4b7c-85d0-c49e75ea582b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:20.667714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1d015-73d4-4b7c-85d0-c49e75ea582b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:20.671219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:20.671817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:20.672089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:20.672382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:20.681569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:20.683035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca604ec-bc67-428e-ab3b-9a6164e7ffd0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:20.692504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 768/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:24.652734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:24.653331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:24.653331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:24.654647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:24.660129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29366c-4493-466f-8b0d-42bf62c90a55 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:24.660633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29366c-4493-466f-8b0d-42bf62c90a55 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:24.660743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29366c-4493-466f-8b0d-42bf62c90a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:24.679861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5f29366c-4493-466f-8b0d-42bf62c90a55 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:24.681053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f29366c-4493-466f-8b0d-42bf62c90a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:24.688905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:25.002609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:25.004406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:25.068490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:54:25.083555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcc3ea-88cb-4444-8f24-ac0c87cd816d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:25.087325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:25.088012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:25.088354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:25.088677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:25.099395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:25.100869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9876d4-5ec2-4a19-a88d-6b842e87f138 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:25.110492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 767/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:27.156989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:27.157550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:27.157550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:27.158863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 17:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:54:27.164695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d1cb-9442-4d5a-8a26-de946d484e0d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:27.165215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d1cb-9442-4d5a-8a26-de946d484e0d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:27.165215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d1cb-9442-4d5a-8a26-de946d484e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:27.184798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3778d1cb-9442-4d5a-8a26-de946d484e0d None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:54:27.185982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3778d1cb-9442-4d5a-8a26-de946d484e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:27.193763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 17:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:54:27.495475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:54:27.497167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:54:27.510659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45208e8e-3a92-4961-8b87-89eac1d6d1a7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:54:27.514258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:54:27.514852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:54:27.515160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:54:27.515431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:54:27.525460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:54:27.526892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adb06c-4c93-4d36-9c7c-b0ee1bda1f37 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:54:27.536670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 771/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:54:37.179460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=11 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:54:37.532365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:57.982124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:57.982124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:57.982124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.001293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.002492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.018330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 769/1540] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.029129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.029776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.029776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.058894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.060323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.072731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 772/1541] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:55:58.078346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.094094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92948e-608d-471b-bd3b-beae443f3a97 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.099425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.100062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.100339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.100611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.101362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:58.101667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.103384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.124293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 770/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:58.132225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.147938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bcd55-c0bc-499a-96c3-742fda5de60c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.156735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.157414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.157737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.158027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.159463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.161777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.223540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 773/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/projects => generated 403 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:58.227751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.243220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5526630-af82-4f29-8dda-3a813b7e27fc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.323016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.323830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.324151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.324425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.325130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.327158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.361410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 771/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:58.365673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.384644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118652b-2a09-4e2b-b01e-2c5ad1fc1dba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.395578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.396199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.396476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.397276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.397455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.399033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfbe02-38f1-4d7f-8d60-f5beb6398534 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.415793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 774/1545] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:58.420113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.434805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818833-2f73-44d5-b3a0-a736255fd833 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.439886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.440674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.440816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.440995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.441128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.441225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.441314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.441842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.445672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53b20a-7ea5-4116-95b3-7295cc1da808 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.469290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 772/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:58.470565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c188df-f1aa-402e-a268-a1160eb528ea None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.470847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c188df-f1aa-402e-a268-a1160eb528ea None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.471246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c188df-f1aa-402e-a268-a1160eb528ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.475231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.476376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.492458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-95c188df-f1aa-402e-a268-a1160eb528ea None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.493741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-95c188df-f1aa-402e-a268-a1160eb528ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.498853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 775/1547] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:55:58.499576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.502849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 773/1548] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.504067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819427-e726-458a-93b9-2555ebdc3650 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.504585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819427-e726-458a-93b9-2555ebdc3650 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.504714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819427-e726-458a-93b9-2555ebdc3650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.517623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e188930-534a-44bc-9fa9-ddf8fde9bd43 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.526656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-09819427-e726-458a-93b9-2555ebdc3650 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.528713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-09819427-e726-458a-93b9-2555ebdc3650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.537121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 774/1549] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.538345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f48eb-b34b-48cf-8009-4c1e0eeb5ba5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.538606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.538762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f48eb-b34b-48cf-8009-4c1e0eeb5ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.538948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f48eb-b34b-48cf-8009-4c1e0eeb5ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.539306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.539585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.539857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] PATH_INFO: `/v3/projects/aae5bfeb043c4cd79096e7b02a3565a1/users/71c8cf5e9cf5404aad15b460e2eaa3d6/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.560486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fa6f48eb-b34b-48cf-8009-4c1e0eeb5ba5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.560835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae5bfeb043c4cd79096e7b02a3565a1, user_id=71c8cf5e9cf5404aad15b460e2eaa3d6, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.561698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fa6f48eb-b34b-48cf-8009-4c1e0eeb5ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.563628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6803a0-9f4a-4cba-ac9c-6de5ac72963d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.570208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 775/1550] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.571486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dd06-4105-45e5-b3e2-76500b5ac735 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.571763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dd06-4105-45e5-b3e2-76500b5ac735 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.572029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811dd06-4105-45e5-b3e2-76500b5ac735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.583500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 776/1551] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:55:58 2022] PUT /identity/v3/projects/aae5bfeb043c4cd79096e7b02a3565a1/users/71c8cf5e9cf5404aad15b460e2eaa3d6/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:55:58.585274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976908-f615-4068-a826-4d53de82b9b6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.585788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976908-f615-4068-a826-4d53de82b9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.586282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75976908-f615-4068-a826-4d53de82b9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.595029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1811dd06-4105-45e5-b3e2-76500b5ac735 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.596301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1811dd06-4105-45e5-b3e2-76500b5ac735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.607700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-75976908-f615-4068-a826-4d53de82b9b6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.608885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-75976908-f615-4068-a826-4d53de82b9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.661357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 776/1552] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.661357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a7bc-d7f3-4bd3-b791-754373e5dd1b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.661357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a7bc-d7f3-4bd3-b791-754373e5dd1b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.661357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7a7bc-d7f3-4bd3-b791-754373e5dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.668474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 777/1553] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.672166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9aa1d-d44b-438d-aa3d-58725f12fb56 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.672166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9aa1d-d44b-438d-aa3d-58725f12fb56 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.672166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9aa1d-d44b-438d-aa3d-58725f12fb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.679011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d9b7a7bc-d7f3-4bd3-b791-754373e5dd1b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.680194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d9b7a7bc-d7f3-4bd3-b791-754373e5dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.688435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 777/1554] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.689220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-47f9aa1d-d44b-438d-aa3d-58725f12fb56 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.689605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980864e-56af-4a8e-9575-446b33d79c66 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.689882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980864e-56af-4a8e-9575-446b33d79c66 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.690154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980864e-56af-4a8e-9575-446b33d79c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.690410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47f9aa1d-d44b-438d-aa3d-58725f12fb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.701816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 778/1555] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.703569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208729eb-fbd0-4ae8-9777-d74e427ebe0b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.703858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208729eb-fbd0-4ae8-9777-d74e427ebe0b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.704134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208729eb-fbd0-4ae8-9777-d74e427ebe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.713238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5980864e-56af-4a8e-9575-446b33d79c66 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.714542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5980864e-56af-4a8e-9575-446b33d79c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.728693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 778/1556] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.728693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.728693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.728693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.729696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-208729eb-fbd0-4ae8-9777-d74e427ebe0b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.730848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-208729eb-fbd0-4ae8-9777-d74e427ebe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.739777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 779/1557] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.740977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.741254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.741518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.747923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.749171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.757880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 779/1558] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.758784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.769377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab26a1-c54d-471f-adb7-409ae7d667cd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.774183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.774337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] MFA Rules not processed for user `71c8cf5e9cf5404aad15b460e2eaa3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.774781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.775097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.775370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.775483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.776013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:58.776316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.778098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.790065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 780/1559] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:58.791110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.803597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38495e7f-387a-4b7b-8382-a976a69faabf admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.808444 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.809081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.809913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.810184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.810825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:58.811262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.812899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d2f42-a4b0-4603-ad51-4d9c05ae1e46 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.814529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:58.826931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 781/1560] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:58.828133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b65c-61b8-4277-9f67-8c297361e658 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.828420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b65c-61b8-4277-9f67-8c297361e658 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.828820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32b65c-61b8-4277-9f67-8c297361e658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.837751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:58.844180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 780/1561] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.845133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.851306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8b32b65c-61b8-4277-9f67-8c297361e658 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.852506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8b32b65c-61b8-4277-9f67-8c297361e658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.856689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cb3e7-6069-48d7-a4d9-31245ed0548b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.860901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 782/1562] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.861270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.861865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.862061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c56e-fd69-45fe-817a-7b52462d6313 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.862265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.862380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c56e-fd69-45fe-817a-7b52462d6313 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.862543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.862643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c56e-fd69-45fe-817a-7b52462d6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.863210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:58.863519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.864614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:55:58.864614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:55:58.864614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:55:58.865691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.879033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 781/1563] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:58.880219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.886817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6a96c56e-fd69-45fe-817a-7b52462d6313 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.888363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6a96c56e-fd69-45fe-817a-7b52462d6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.889987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bd3f8-27a1-4c2a-be18-dea11cc954ae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.894521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.895133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.895417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.895690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.896210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 783/1564] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.896348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:58.896647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.897357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.897663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.897939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.898375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.911597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 782/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:58.912229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.916807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:55:58.917995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:55:58.922649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf2fd-8296-46b7-9ab7-6cfaf56da1d8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.926573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 784/1566] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:55:58.927437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.931166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.931765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.932051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.932325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.932942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.934918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.937199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a31ea-f570-4152-b0d7-db1f323b26b8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:58.941860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:58.942443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:58.942721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:58.943021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:58.943643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:58.944650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:55:58.944650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:55:58.944650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:55:58.945830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:58.966839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 783/1567] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:58.966839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:58.975071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 785/1568] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:58.975071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.029354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d93826-3c99-41ff-8b5c-b6f217a07ed3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.034172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.034783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.035095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.035372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.035997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.037947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.038464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7809f14-3311-47cc-8fc1-eb076931aa1f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.043023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.043708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.043986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.044251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.044896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:59.045196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.047736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.071186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 786/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:59.071186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.075817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 784/1570] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:58 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.076810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.084148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd1bd4-c40f-42ce-b017-28ad2f30ac2f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.140964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258167b4-c655-4b1a-9be3-e1313b05c217 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.145795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.146154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.146392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.146668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.146761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.146935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.147069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.147303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.147576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.147976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:59.148243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.149470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.150321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.163813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 785/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:59.164516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.176516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a034d0-2dd6-4e05-84b1-fc62a57a3ced admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.181121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.181710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.181988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.182256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.187724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 787/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.187927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.188029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.188131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.216219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 786/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.216219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.222356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.223067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.253642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eff28a-54a5-41f5-b9ed-5b2b6722fe06 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.279828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f72d8-2ba1-47e4-9d0d-d2228cdcb4e1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.284269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.284863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.285138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.285402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.286042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.287079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:55:59.287079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:55:59.287079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:55:59.288272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.313181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 787/1574] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.313181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.314199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.314837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.315147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.315474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.316131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:55:59.316488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.318152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.320596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f0099-44dd-4b27-b21f-a5e2252dfa2c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.326048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.326388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.326663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.326925 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.327559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.329464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.330628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 788/1575] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:55:59.331555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.341617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecbf54-e156-4a07-9d90-fe966e200e8a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.346847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.347465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.347739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.348004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.349319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.354348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 788/1576] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.354506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.354609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.362554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f3b45-5a00-444e-b509-88ed5db4b6ea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.367438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.368028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.368332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.368683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.369358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.371824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.382082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 789/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.382878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.402197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 789/1578] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 308 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.402197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.410543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f0d-1ccb-489a-8fde-9968e01868e0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.446057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ec1cd-9458-4f1d-b5fc-e041957d3fde admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.450825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.452672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.452672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.452672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.452672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.454901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.475469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.476203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.476487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.476752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.477394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.479366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.487763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 790/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.487763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.503458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 790/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.504543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.514217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bc6fd-1b31-4162-ac86-a5b98ccf8175 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.530299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.554808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c4da5-3c23-4aa0-9320-edb54478e57a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.560125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.560728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.561008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.561279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.561892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.563393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.563843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.575415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.576441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.576785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.576951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.577843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.579455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.584933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 791/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.585823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.592335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 791/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.595409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87c051-35aa-44c0-a74e-e8fc4dd80cd0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.595983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.600164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.600950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.601321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.601715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.602424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.604476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.605562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6920-f55f-4b79-b264-3ed2322cc7b3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.611409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.612008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.612291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.612564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.613238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.614834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.626396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 792/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.627679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.636406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 792/1584] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.637009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070adca1-1f95-4f44-b21c-022af255fe4a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.638257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.697315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.697985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.698446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.698754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.699442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.701986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.702369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa7691-fd82-4c2f-b645-60a1f2793149 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.707871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.708795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.709140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.709478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.710147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.711946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.728356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 793/1585] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.728356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.728672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 793/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.728672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.739644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f90cd8-c140-4fd8-82c9-38d6f8da44fd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.739892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67815f6d-cd54-4470-968b-57c4c312714d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.744776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.745007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.745625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.745736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.745903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.746173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.746269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.747413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.747654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.748273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.749377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.751368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.761485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 794/1587] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.763058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.774862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45f746-a48d-474a-a919-e2207dcaa52b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.781128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 794/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.781128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.781417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.781417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.781417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.781417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.781781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.783702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.790144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d4cb2-d80f-4cf5-a50b-caae0ff10587 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.795056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.795799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 795/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.795973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.796138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.796290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.796734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.796950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.798584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.807130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760fb95-641b-4368-9c15-48110efdf936 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.811182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 795/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.812026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.812999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.813617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.813903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.814244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.814890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.816545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.822231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977be784-bb75-4a5a-aa0f-9664ca2ff827 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.828134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 796/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.828313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.829021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.829306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.829474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.829578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.830199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.832634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.839827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd112-0b9e-4038-b3c4-e04a84e74319 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.844848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.845574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.846143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.846647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] PATH_INFO: `/v3/projects/92a20a94f9264926b14f330856c89114/users/4b11d1cbe0134935b98e9384b1fa4982/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.859674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 796/1592] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:55:59 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:55:59.859674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.866746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a20a94f9264926b14f330856c89114, user_id=4b11d1cbe0134935b98e9384b1fa4982, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.866998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66f4a-e49d-415d-a9f4-e54a6eaf55c7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.869617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.872374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.872955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.873231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.873505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.874338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.876314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.884354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 797/1593] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:55:59 2022] PUT /identity/v3/projects/92a20a94f9264926b14f330856c89114/users/4b11d1cbe0134935b98e9384b1fa4982/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:55:59.885204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.890351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 797/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.890351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.898354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361a66e-12e8-415d-ac5c-51b5dc067834 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.939666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.949303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e3722d-3639-4aa9-b465-d8a13f342ec7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.955021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.955623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.955901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.956168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.956808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.958393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.958851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.959489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.959768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.960038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.960678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.962266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.968353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 798/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.969153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.974953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:55:59.978021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 798/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:55:59.978021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:55:59.979287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69979b97-a074-4f55-a4bd-5b914ff4d95a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.985815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.986406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.986679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.986944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.987604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:55:59.987821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ed6aa-3c20-46b0-9978-ed4c1f72a513 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:55:59.989181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:55:59.992547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:55:59.993193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:55:59.993473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:55:59.993750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] PATH_INFO: `/v3/projects/6816355a42d64d5988aff83532ff5888/users/cb11c04acb4f40cd9d09edf16b11abb0/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:55:59.999940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 799/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.000704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.009807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6816355a42d64d5988aff83532ff5888, user_id=cb11c04acb4f40cd9d09edf16b11abb0, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.010347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f1e1-32c4-4a99-8343-fff37df9268e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.010838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:56:00.010838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:56:00.010838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:56:00.013463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.015937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.016615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.016890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.017160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] PATH_INFO: `/v3/projects/f9182ab0c79d4b8a9de4c319491ca316/users/478a0e3ac431440582d3dca1fe246ac3/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.028429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 799/1598] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:55:59 2022] PUT /identity/v3/projects/6816355a42d64d5988aff83532ff5888/users/cb11c04acb4f40cd9d09edf16b11abb0/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.029303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.033694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9182ab0c79d4b8a9de4c319491ca316, user_id=478a0e3ac431440582d3dca1fe246ac3, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.036489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.050166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 800/1599] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:55:59 2022] PUT /identity/v3/projects/f9182ab0c79d4b8a9de4c319491ca316/users/478a0e3ac431440582d3dca1fe246ac3/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.051083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.098826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95273dd7-7995-4a3f-8273-39784c4cf041 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.113587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e548-c4e1-4721-afa7-c072657ad407 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.118202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.118799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.119108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.119387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.120192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.122631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.132603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 801/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.133399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.142542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e65a7e-6295-44b0-baea-5df9d26059fd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.147251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.147809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.148088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.148355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.148976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.150796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.159585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.160195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.160476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.160749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.160911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 802/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.161532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.163158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.164007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.176159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 800/1602] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.176756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b53458-36f3-4016-a98b-585749196ade admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.177369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.183133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.183779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.184082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.184383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] PATH_INFO: `/v3/projects/d2d3d0d44ab748d5bcdc09452916b6bc/users/553aa53db51b4c8aaac26e81f0521f04/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.187883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067a563-95bf-418f-a9d9-7dd9d57d69eb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.192690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.193299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.193575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.193874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.194521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.196156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.202316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d3d0d44ab748d5bcdc09452916b6bc, user_id=553aa53db51b4c8aaac26e81f0521f04, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.205552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4d754-d911-43d7-b273-516e36ee45eb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.209770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 801/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.210660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.222585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f90cf-0f4b-45d9-82d0-38000a775826 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.223445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 803/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/d2d3d0d44ab748d5bcdc09452916b6bc/users/553aa53db51b4c8aaac26e81f0521f04/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.224986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c75-4e56-447c-b6e4-ef00091e1813 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.225274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c75-4e56-447c-b6e4-ef00091e1813 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.225551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c75-4e56-447c-b6e4-ef00091e1813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.249079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-69522c75-4e56-447c-b6e4-ef00091e1813 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.250254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-69522c75-4e56-447c-b6e4-ef00091e1813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.276307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:00.292448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.293120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.293433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.293702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] PATH_INFO: `/v3/projects/13cf0de2979c44a08f500430834617d2/users/f7491479bd88425f8234e0b1d0e11e55/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.307904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cf0de2979c44a08f500430834617d2, user_id=f7491479bd88425f8234e0b1d0e11e55, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.310599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.311450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 804/1605] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.312500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.312778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.313061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.327661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 802/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/13cf0de2979c44a08f500430834617d2/users/f7491479bd88425f8234e0b1d0e11e55/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.328602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.336229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.337425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.364367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:00.391639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce547136-7588-4158-a543-d14b76226b99 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.396318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.396993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.397270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.397541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] PATH_INFO: `/v3/projects/92a20a94f9264926b14f330856c89114/users/4b11d1cbe0134935b98e9384b1fa4982/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.399441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a20a94f9264926b14f330856c89114, user_id=4b11d1cbe0134935b98e9384b1fa4982, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.399616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 805/1607] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.400239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.402096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.411710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83bd71-10e3-4f06-aefe-3f608b000f0e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.412555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 803/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/92a20a94f9264926b14f330856c89114/users/4b11d1cbe0134935b98e9384b1fa4982/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.413358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.475998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.476682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.476965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.477334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] PATH_INFO: `/v3/projects/83003e8d88df45269833e6aa8b5d44cf/users/9dde01ad0b5d4caf932a3fd9517e6956/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.484073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edc109-eca0-4b66-9f92-cde321db8102 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.489517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.490250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.490577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.491008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.491784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.492781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83003e8d88df45269833e6aa8b5d44cf, user_id=9dde01ad0b5d4caf932a3fd9517e6956, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.493598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.496636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47305ce9-e44d-4e58-afa3-7c55297674e6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.511413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 804/1609] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:00.512900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.523347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 806/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/83003e8d88df45269833e6aa8b5d44cf/users/9dde01ad0b5d4caf932a3fd9517e6956/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.523347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.523347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.523347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.533076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e466bb5-5e58-4b35-82ea-5066d16adecc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.552983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] MFA Rules not processed for user `553aa53db51b4c8aaac26e81f0521f04`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.554113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.591952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.592604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.592878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.593153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] PATH_INFO: `/v3/projects/dc7eada23fab4b428e8dd7be1738cea5/users/e18740d304014f8caa523b5d41500140/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.595317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:00.609585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7eada23fab4b428e8dd7be1738cea5, user_id=e18740d304014f8caa523b5d41500140, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.612373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7741b-6c01-43c0-92dd-4961619aada3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.621103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 807/1611] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.623232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.627555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 805/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/dc7eada23fab4b428e8dd7be1738cea5/users/e18740d304014f8caa523b5d41500140/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.628799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.629083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.629353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.649479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.650676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.677476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:00.689372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1285ae-25a3-433f-b4b9-a302e519553b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.712516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 806/1613] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.715836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.739276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.739887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.740165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.740439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.750299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.751800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fb78c-0394-450f-84ed-3525179ca476 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.761638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 808/1614] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:00.762958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fe657-c98e-493e-8f08-e742f2c839cb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.763276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fe657-c98e-493e-8f08-e742f2c839cb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.763554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fe657-c98e-493e-8f08-e742f2c839cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.768717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09984779-48d6-4851-b757-298b3fe1faaa None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.772493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.773102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.773381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.773654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.783552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.785022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.794533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8b0fe657-c98e-493e-8f08-e742f2c839cb None None] MFA Rules not processed for user `4b11d1cbe0134935b98e9384b1fa4982`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.795867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8b0fe657-c98e-493e-8f08-e742f2c839cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.796132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 807/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:00.797332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.807195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510fba4-78f1-4a94-916a-880ed59c934f service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:00.811977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:00.812694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.813049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.813399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] PATH_INFO: `/v3/projects/3b320e561ec5450682ed979963c392a0/users/18bf10b172e7450a9b618683643ed48e/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.830007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b320e561ec5450682ed979963c392a0, user_id=18bf10b172e7450a9b618683643ed48e, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:00.832819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e6fb5-3c8a-43ed-b203-8f055fa643e2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:00.853849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 808/1616] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:00 2022] PUT /identity/v3/projects/3b320e561ec5450682ed979963c392a0/users/18bf10b172e7450a9b618683643ed48e/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:00.853849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cb117-1c6f-4e30-97a1-37d57e19eb04 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.853849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cb117-1c6f-4e30-97a1-37d57e19eb04 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.853849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cb117-1c6f-4e30-97a1-37d57e19eb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.884342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 809/1617] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.886175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a073-7dfa-4729-bd69-b8b00ed22107 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.886566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a073-7dfa-4729-bd69-b8b00ed22107 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.887094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a073-7dfa-4729-bd69-b8b00ed22107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.891267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c26cb117-1c6f-4e30-97a1-37d57e19eb04 None None] MFA Rules not processed for user `9dde01ad0b5d4caf932a3fd9517e6956`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.892048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c26cb117-1c6f-4e30-97a1-37d57e19eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.919415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f456a073-7dfa-4729-bd69-b8b00ed22107 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:00.920643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f456a073-7dfa-4729-bd69-b8b00ed22107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.957104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:00.981485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 809/1618] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.982909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.983446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.983889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:00.985402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 810/1619] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 9 17:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:00.986234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:00.988247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:00.998071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 810/1620] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:00.999427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:00.999709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:00.999979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.021286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:01.022502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:01.035035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 811/1621] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:01.035035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.059232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da270f68-d71a-4939-808e-f953adc36ab3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.064260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:01.074063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.075238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.078290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.078290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.086835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.088325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.098773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 811/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:01.099655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.110193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a8486-d4c6-46a7-afd7-b76f690ac292 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.111218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fae0f-ae2a-4833-b248-8aa91d2a5c4f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.115430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.115724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.116027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.116306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.116578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.116805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.117190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.117305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.117629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.119422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787765c-8ac4-4f73-9e33-5667697819aa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.173706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 812/1623] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:01 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:01.175307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.175932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.176312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.216924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.218528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.237162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 812/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:01.238721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.255279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2dc938-5563-4107-a097-591877f3e663 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.265129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:01.267072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:01.329001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.329697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.330236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.330618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.415536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 813/1625] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 9 17:56:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:01.416467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.429177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db89d5-c59e-4c83-ba9e-a9c8b16157e5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.432387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.434745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.443784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.444698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.445166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.445585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.462298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 813/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:01.475688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.483821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.486546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.499114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9dad-f002-429f-b54c-89892dc52c92 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.510784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 814/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:01.512290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.514921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.515730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.517140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.517496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.518531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.522540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.551362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98ad4f-19cb-452b-8d85-560350dea70a service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.563724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.565132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.566384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.566821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.580584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 814/1628] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:01 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:01.581681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.590217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:01.595362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779967b-2ae3-4212-b633-fb202b3845c3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.603449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.604236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.604717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.605123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.614792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.617312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.636986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 815/1629] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:01.639125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.648732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:01.654484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ee462-1f1b-42a4-9fc2-92dc7e0ce153 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.663804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.664944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.665561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.666172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] PATH_INFO: `/v3/users/71c8cf5e9cf5404aad15b460e2eaa3d6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.667947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c8cf5e9cf5404aad15b460e2eaa3d6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.671727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.682716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.685093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.712320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 815/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:01.713363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.740998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92cee24-d415-4b5e-94fa-168af4cdd08f service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.757548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.758235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.758624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.758905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] PATH_INFO: `/v3/projects/175e9b72ea604a878b3fadc7ee98c45f/users/0715f4d34b5b4fa7a4758351b93b3ea8/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.784460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e9b72ea604a878b3fadc7ee98c45f, user_id=0715f4d34b5b4fa7a4758351b93b3ea8, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.787982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.790254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c8cf5e9cf5404aad15b460e2eaa3d6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:01.808511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 816/1631] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:01 2022] PUT /identity/v3/projects/175e9b72ea604a878b3fadc7ee98c45f/users/0715f4d34b5b4fa7a4758351b93b3ea8/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:01.808960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c8cf5e9cf5404aad15b460e2eaa3d6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:01.810010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.810213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c8cf5e9cf5404aad15b460e2eaa3d6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:01.813429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:01.862587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c8cf5e9cf5404aad15b460e2eaa3d6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:01.894302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adbe1e-52f9-4c1d-8eb8-417c78786489 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.897787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-472cfd78-284a-419f-b36e-c943f4dd0537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c8cf5e9cf5404aad15b460e2eaa3d6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:01.899417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:01.900037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.900313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.900581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.901287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:01.903506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:01.919154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 817/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:01.919378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 816/1633] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:01 2022] DELETE /identity/v3/users/71c8cf5e9cf5404aad15b460e2eaa3d6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:01.921904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:01.922435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e103-4f07-4616-ae97-1a5f37bfbf90 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:01.922726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e103-4f07-4616-ae97-1a5f37bfbf90 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:01.923015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595e103-4f07-4616-ae97-1a5f37bfbf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:01.935570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34ac94-aa6d-43b0-9b49-02e631b1e094 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:01.959057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0595e103-4f07-4616-ae97-1a5f37bfbf90 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:01.960427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0595e103-4f07-4616-ae97-1a5f37bfbf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.008692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.009389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.009668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.009937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] PATH_INFO: `/v3/projects/175e9b72ea604a878b3fadc7ee98c45f/users/0715f4d34b5b4fa7a4758351b93b3ea8/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.012024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175e9b72ea604a878b3fadc7ee98c45f, user_id=0715f4d34b5b4fa7a4758351b93b3ea8, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.015394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721681f5-b436-47d8-9c9a-a5f32341aec2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.032719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 818/1634] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:01 2022] PUT /identity/v3/projects/175e9b72ea604a878b3fadc7ee98c45f/users/0715f4d34b5b4fa7a4758351b93b3ea8/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:02.043097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19801-8149-4d10-b73e-a14d68036f1c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.043385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19801-8149-4d10-b73e-a14d68036f1c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.044603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d19801-8149-4d10-b73e-a14d68036f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.089902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 817/1635] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:02.102309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.102910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.103494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.105677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 818/1636] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 9 17:56:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:02.107076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.107501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-57d19801-8149-4d10-b73e-a14d68036f1c None None] MFA Rules not processed for user `0715f4d34b5b4fa7a4758351b93b3ea8`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:02.108709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-57d19801-8149-4d10-b73e-a14d68036f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.109958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.150726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:02.200200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb0e14-950a-4f7e-9740-4acf8248cdb5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.204748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 819/1637] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:02.206689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.207100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.207272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.210057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.210857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.211477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.212065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.245780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:02.248678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:02.250063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.347447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.349011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.364146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 820/1638] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:02.370552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 819/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:02.382743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.383694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.384064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.386185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:02.526415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c3fa-d3bb-4fa4-9dd7-89e35cab3b8e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.534065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.534747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.535052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.535799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.555828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb960f3-6c56-473c-b2a2-ebb2cf450c4f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.560986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.562488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.567417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.568467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.569999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.570624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.588964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:02.589806 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 821/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:02.611989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:02.664016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 820/1641] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:02.862038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:02.879380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aae82-c56a-43bd-b8a2-a459aa87779f service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:02.893422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:02.894716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:02.895277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:02.895728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:02.900052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:03.054265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:03.056564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:03.066009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:03.075648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 822/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:03.326810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:03.329150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:03.358004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b8b6e-77ad-4343-91d0-f5abb5c9f1cd service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:03.374732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:03.375439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:03.375723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:03.376006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] PATH_INFO: `/v3/projects/aae5bfeb043c4cd79096e7b02a3565a1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:03.377297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae5bfeb043c4cd79096e7b02a3565a1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:03.378336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:56:03.378336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Aug 09 17:56:03.378336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: Aug 09 17:56:03.379606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:03.515150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae5bfeb043c4cd79096e7b02a3565a1', 'request_id': 'req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.526031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae5bfeb043c4cd79096e7b02a3565a1', 'request_id': 'req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.542933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae5bfeb043c4cd79096e7b02a3565a1', 'request_id': 'req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.564327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] The token cache is being invalidate because project aae5bfeb043c4cd79096e7b02a3565a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:03.564542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.568204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8f8e0ba4-5959-4608-acc4-e0b03e3ebcf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.580652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 821/1643] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:03 2022] DELETE /identity/v3/projects/aae5bfeb043c4cd79096e7b02a3565a1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:03.585127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:03.605818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41f19-70de-4448-ab58-e537bfab2f2b service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:03.726339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:03.727315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:03.727784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:03.728227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] PATH_INFO: `/v3/users/4b11d1cbe0134935b98e9384b1fa4982` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:03.729919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b11d1cbe0134935b98e9384b1fa4982)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:03.731412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: WARNING py.warnings [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 17:56:03.731412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: warnings.warn(msg) Aug 09 17:56:03.731412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: Aug 09 17:56:03.733180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:03.817206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b11d1cbe0134935b98e9384b1fa4982'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.832914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b11d1cbe0134935b98e9384b1fa4982'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.834138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b11d1cbe0134935b98e9384b1fa4982'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.846846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b11d1cbe0134935b98e9384b1fa4982'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.882879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-307a26df-405b-4136-a6f9-015b0bc4b42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b11d1cbe0134935b98e9384b1fa4982'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:03.893113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 823/1644] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:03 2022] DELETE /identity/v3/users/4b11d1cbe0134935b98e9384b1fa4982 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:05.414019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956272-a4dd-484b-8654-88e09961f877 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.414649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956272-a4dd-484b-8654-88e09961f877 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.414649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956272-a4dd-484b-8654-88e09961f877 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.416458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 822/1645] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 9 17:56:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:05.423332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6374ff-e85b-4fd5-ba9d-3963e8fabe25 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.423868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6374ff-e85b-4fd5-ba9d-3963e8fabe25 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.423868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6374ff-e85b-4fd5-ba9d-3963e8fabe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.475246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bc6374ff-e85b-4fd5-ba9d-3963e8fabe25 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:05.476554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bc6374ff-e85b-4fd5-ba9d-3963e8fabe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:05.619198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 824/1646] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 9 17:56:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:05.627037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.627309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.627584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.629398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 823/1647] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 9 17:56:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:05.635657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.636612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.637100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.664535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:05.666223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:05.719258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:05.720137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 825/1648] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:56:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:05.721831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:05.810511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:05.812111 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:05.821410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:05.866372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:05.897493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce57e7-40cb-4618-9bbe-74b0c4dbd95d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:05.904937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:05.906204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:05.907788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:05.908250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:05.929073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:05.931095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6dc999-f269-4a7c-81ee-df1fc4a4db37 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:05.993237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90f5e-ff53-4e48-8688-bdd122b3e333 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:06.021225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 824/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:06.027350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:06.028995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.031642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.036737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] PATH_INFO: `/v3/projects/92a20a94f9264926b14f330856c89114` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.040447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a20a94f9264926b14f330856c89114)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:06.044902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:06.204456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a20a94f9264926b14f330856c89114', 'request_id': 'req-fc6a5189-3658-4843-86ec-ea50c984c989'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:06.211138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a20a94f9264926b14f330856c89114', 'request_id': 'req-fc6a5189-3658-4843-86ec-ea50c984c989'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:06.225253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a20a94f9264926b14f330856c89114', 'request_id': 'req-fc6a5189-3658-4843-86ec-ea50c984c989'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:06.282154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] The token cache is being invalidate because project 92a20a94f9264926b14f330856c89114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:06.282154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:06.282154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fc6a5189-3658-4843-86ec-ea50c984c989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:06.282154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 826/1650] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:05 2022] DELETE /identity/v3/projects/92a20a94f9264926b14f330856c89114 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:06.284375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.284375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.284375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.323310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:06.324569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:06.468492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.469541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.469970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.479065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 825/1651] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:06.507471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:06.509414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:06.526333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:06.537601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 827/1652] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:06.538934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e124f-0ada-4bb4-b5be-027e722fe2b3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:06.553816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:06.554765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.555262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.555675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.556488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:06.556936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:06.560538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:06.599042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:06.609481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fb1f9-e20f-4c52-bd9a-be74039a6bc7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:06.612470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 826/1653] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:06.621002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:06.623298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.624851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.625306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.626154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:06.631297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:06.697479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:06.708431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 828/1654] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:06 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:06.713390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206042f-0afc-424e-b237-1a00ef07d405 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:06.888488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:06.890551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:06.891066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:06.891542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:06.892702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:06.897307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:06.986315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 827/1655] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:06 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:06.987402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:06.999569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e583-2f14-44a4-a13d-f9a8cec57aeb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.036540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.037172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.037560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.037731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.038385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.040090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.079650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.096878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 829/1656] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:07.117472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587e721-7d90-4506-a6c7-b4ad30a6aa24 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.145833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.149679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.151105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.153479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.165490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.186740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd39e32-b666-43e9-a654-d58b4030269c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.238834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.241185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 828/1657] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:07.249736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6de7c-e114-429f-b385-06fd9d4b1301 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.276943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.277632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.277949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.278224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] PATH_INFO: `/v3/projects/b41b4d5f87fd4a48b4c8e8d1c40d1e50/users/125f71f3e47043ae905948146b485088/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.306129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41b4d5f87fd4a48b4c8e8d1c40d1e50, user_id=125f71f3e47043ae905948146b485088, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.308869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.346507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 830/1658] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:07 2022] PUT /identity/v3/projects/b41b4d5f87fd4a48b4c8e8d1c40d1e50/users/125f71f3e47043ae905948146b485088/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:07.361469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.362104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.362800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.526408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] MFA Rules not processed for user `125f71f3e47043ae905948146b485088`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:07.529739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:07.710297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.712244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:07.734397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.777534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:07.834798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 829/1659] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:07.842850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:07.845888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb4f60-dac3-4743-89d0-fdecf9422c41 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.856883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.857527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.857969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.858413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] PATH_INFO: `/v3/users/9dde01ad0b5d4caf932a3fd9517e6956` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.869535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99a5c5-e023-47b1-9241-31395205c526 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:07.872723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dde01ad0b5d4caf932a3fd9517e6956)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.874697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:07.886277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:07.887977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:07.889469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:07.890189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:07.891613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:07.897865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.027034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dde01ad0b5d4caf932a3fd9517e6956'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:08.036302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 830/1660] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:07 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:08.040330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.094452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dde01ad0b5d4caf932a3fd9517e6956'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:08.095911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dde01ad0b5d4caf932a3fd9517e6956'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:08.129034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dde01ad0b5d4caf932a3fd9517e6956'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:08.172367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d0b09de1-a46b-49e9-aba8-d188754c4269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dde01ad0b5d4caf932a3fd9517e6956'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:08.190422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 831/1661] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:07 2022] DELETE /identity/v3/users/9dde01ad0b5d4caf932a3fd9517e6956 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:08.196992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.198354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.198862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.240736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c4424-af9b-4ffd-a742-d1d6e85dae06 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.281823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:08.285798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:08.367313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.367944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.368284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.368686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.369638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.371855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.423699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 831/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:08 2022] POST /identity/v3/users => generated 293 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:08.430814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.432651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:08.458255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18865db-ef41-4c84-a9dc-acc80e1b5372 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.472886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.473518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.473797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.474072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.474725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.476389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.508516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 832/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:08.510178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.529904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 832/1664] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:08.537694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56035dc0-9b04-4ac8-b7a6-e78c64deadf1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.550785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.552352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.552838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.553328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.554401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.556286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.571100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 833/1665] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:08.576126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.591333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecfbea-822e-4259-9a1b-b88acf502cf5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.607864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:08.610099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:08.619206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:08.620601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.623635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.624155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] PATH_INFO: `/v3/projects/79d9b2bb72044816aa6e80b443a76d1d/users/25c4209c251a4e7c9cd75cd51cefa1a3/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.668950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d9b2bb72044816aa6e80b443a76d1d, user_id=25c4209c251a4e7c9cd75cd51cefa1a3, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:08.680210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf09013-29ce-4a23-9b16-645f151fc47f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:08.698116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:08.718757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 833/1666] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:08 2022] PUT /identity/v3/projects/79d9b2bb72044816aa6e80b443a76d1d/users/25c4209c251a4e7c9cd75cd51cefa1a3/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:08.728930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:08.729590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:08.730064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:08.810212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f52bef-6a7c-4b6c-8ac1-f96ddf8b79be admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:08.848451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] MFA Rules not processed for user `25c4209c251a4e7c9cd75cd51cefa1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:08.856971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:08.885074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:09.038684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.039295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.039579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.046882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 834/1667] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:09.048352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.050821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.223572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.225521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.236941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a436fb3-9d4d-483c-bc46-c84a6cb2986c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.250305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.251401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.251401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.251905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.252347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.254905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.257405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 834/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:09.306448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.307504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 835/1669] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:09 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:09.318373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb2c14-99ea-4a5c-bd3a-72a81464fc28 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.429963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.430688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.431501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.431821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.433159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.435583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.474593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.485352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 835/1670] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:09 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:09.489704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dff007-ec74-44c8-bacc-e6a9baadb396 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.496315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.497587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.497874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.498319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.499008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.501294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.521390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.536209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 836/1671] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:09.539187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f426-2f8d-4475-bfe7-cfbb0f9aff6b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.544536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.545252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.545947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.546441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] PATH_INFO: `/v3/projects/fd6c9f530a9c4423a8521e4971095f88/users/3b34ec4cd3234daf802ef82ad3f1802f/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.575263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6c9f530a9c4423a8521e4971095f88, user_id=3b34ec4cd3234daf802ef82ad3f1802f, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.580259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.612323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.613502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 836/1672] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:09 2022] PUT /identity/v3/projects/fd6c9f530a9c4423a8521e4971095f88/users/3b34ec4cd3234daf802ef82ad3f1802f/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:09.629141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f8769-a591-4638-9f65-bb5a2fcdbf36 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.720995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.724571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.725023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.725592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.726701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.728888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c62c7-772f-40b4-acab-17a4b341af12 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.749212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:09.757902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:09.762340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 837/1673] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:09.764754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a16ea-c249-4739-adc0-29fdf55e7783 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:09.774727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:09.775776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.776201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.776617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] PATH_INFO: `/v3/projects/fd6c9f530a9c4423a8521e4971095f88/users/3b34ec4cd3234daf802ef82ad3f1802f/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.780323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6c9f530a9c4423a8521e4971095f88, user_id=3b34ec4cd3234daf802ef82ad3f1802f, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:09.783630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:09.815088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 837/1674] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:09 2022] PUT /identity/v3/projects/fd6c9f530a9c4423a8521e4971095f88/users/3b34ec4cd3234daf802ef82ad3f1802f/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:09.816910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:09.817390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:09.817827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:09.856978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] MFA Rules not processed for user `3b34ec4cd3234daf802ef82ad3f1802f`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:09.858356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:09.875639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:09.989795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 838/1675] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:10.002888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:10.005019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:10.158812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d243067-321b-4ef7-81e2-ce20c24c8509 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:10.166207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:10.166991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:10.167572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:10.167952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:10.179985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:10.184824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:10.257755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 838/1676] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:11.553908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:11.577087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9657544-e630-4d59-9bc3-cc4b884f1a66 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:11.596220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:11.596220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:11.596220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:11.596220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:11.607363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:11.817179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:11.818652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:11.854324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 839/1677] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:12.601880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:12.604594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:12.791447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72059c41-c3fa-4247-b9c5-5815612cd08b service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:12.818283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:12.819024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:12.819314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:12.819605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] PATH_INFO: `/v3/projects/83003e8d88df45269833e6aa8b5d44cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:12.824981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83003e8d88df45269833e6aa8b5d44cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:12.829765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:12.989356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83003e8d88df45269833e6aa8b5d44cf', 'request_id': 'req-7d111f33-821a-45a3-96cd-de68b8210329'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.013528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83003e8d88df45269833e6aa8b5d44cf', 'request_id': 'req-7d111f33-821a-45a3-96cd-de68b8210329'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.022290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83003e8d88df45269833e6aa8b5d44cf', 'request_id': 'req-7d111f33-821a-45a3-96cd-de68b8210329'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.068517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] The token cache is being invalidate because project 83003e8d88df45269833e6aa8b5d44cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:13.069153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.071369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7d111f33-821a-45a3-96cd-de68b8210329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.073939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 839/1678] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:12 2022] DELETE /identity/v3/projects/83003e8d88df45269833e6aa8b5d44cf => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:13.078657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:13.090804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13489-498c-4eca-97db-cd1007f25b0f service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:13.250890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:13.253658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:13.254159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:13.254857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] PATH_INFO: `/v3/users/0715f4d34b5b4fa7a4758351b93b3ea8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:13.260746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0715f4d34b5b4fa7a4758351b93b3ea8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:13.266168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:13.380752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0715f4d34b5b4fa7a4758351b93b3ea8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.387153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:13.394888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0715f4d34b5b4fa7a4758351b93b3ea8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.396470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0715f4d34b5b4fa7a4758351b93b3ea8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.416315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0715f4d34b5b4fa7a4758351b93b3ea8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.441038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0715f4d34b5b4fa7a4758351b93b3ea8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:13.457238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 840/1679] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:13 2022] DELETE /identity/v3/users/0715f4d34b5b4fa7a4758351b93b3ea8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:14.188164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.194146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.195184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.292638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] MFA Rules not processed for user `cb11c04acb4f40cd9d09edf16b11abb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:14.294544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:14.341691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:14.494517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 840/1680] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:14.511296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:14.514125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:14.568888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:14.570320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:14.709757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83e813-5de4-47ab-92c1-7e81833367c6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:14.715892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:14.716548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.716802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8dbf3-5de5-4040-819e-de388c7a934a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:14.716916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.717070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] PATH_INFO: `/v3/users/553aa53db51b4c8aaac26e81f0521f04` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.724104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=553aa53db51b4c8aaac26e81f0521f04)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:14.725820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:14.734689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:14.735344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.735793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.736075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.748165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:14.749946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f89049-b7ee-4919-9b6a-04848994d11d service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:14.770153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 841/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:14.836175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553aa53db51b4c8aaac26e81f0521f04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:14.854373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553aa53db51b4c8aaac26e81f0521f04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:14.855037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553aa53db51b4c8aaac26e81f0521f04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:14.880089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553aa53db51b4c8aaac26e81f0521f04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:14.895289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553aa53db51b4c8aaac26e81f0521f04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:14.919992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 841/1682] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:14 2022] DELETE /identity/v3/users/553aa53db51b4c8aaac26e81f0521f04 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:14.921849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04922fc-06ee-4e5f-90f8-3ea23a08f075 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:14.922489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04922fc-06ee-4e5f-90f8-3ea23a08f075 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:14.923001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04922fc-06ee-4e5f-90f8-3ea23a08f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:14.958543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b04922fc-06ee-4e5f-90f8-3ea23a08f075 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:14.962432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b04922fc-06ee-4e5f-90f8-3ea23a08f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.046475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.048166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.120675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.122136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 842/1683] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:15.197442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2d96-5237-4304-bc09-f0677f8c26fc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.214033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.214654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.214941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.215928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] PATH_INFO: `/v3/projects/175e9b72ea604a878b3fadc7ee98c45f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.217164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175e9b72ea604a878b3fadc7ee98c45f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.220653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.336267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175e9b72ea604a878b3fadc7ee98c45f', 'request_id': 'req-eafdd7d8-a87e-498d-9de8-6431727ef2d9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:15.355603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175e9b72ea604a878b3fadc7ee98c45f', 'request_id': 'req-eafdd7d8-a87e-498d-9de8-6431727ef2d9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:15.364179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175e9b72ea604a878b3fadc7ee98c45f', 'request_id': 'req-eafdd7d8-a87e-498d-9de8-6431727ef2d9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:15.389126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] The token cache is being invalidate because project 175e9b72ea604a878b3fadc7ee98c45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:15.389284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:15.390058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:15.392881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 842/1684] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:15 2022] DELETE /identity/v3/projects/175e9b72ea604a878b3fadc7ee98c45f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:15.398856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201df0-9b3c-4124-a979-4427796e946b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.399509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201df0-9b3c-4124-a979-4427796e946b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.399984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201df0-9b3c-4124-a979-4427796e946b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.441213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-be201df0-9b3c-4124-a979-4427796e946b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:15.445717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.445877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-be201df0-9b3c-4124-a979-4427796e946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.447284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.536209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.561267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 843/1685] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:15.569087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.569457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.569734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.598229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:15.600082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.614473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 844/1686] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:15.616149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd7d8-a87e-498d-9de8-6431727ef2d9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.619345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.632647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f0352-49d0-41ee-ae97-016d5627797b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.636163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.636759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.637145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.637596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.640463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:15.648123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.649244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.649545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.650373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.651464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:15.651572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.658491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.679648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 845/1687] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:15.682760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.688174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.692817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.698242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac71ec4-85b6-4ebe-b7d8-c4c98f530682 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.704264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.705090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.705578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.705826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.706556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.709413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.724317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 843/1688] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:15.758305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 846/1689] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:15 2022] POST /identity/v3/projects => generated 407 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:15.762609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:15.775166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10a2bd-ceb4-4c46-9f22-def9190f00d5 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:15.912539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:15.913330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:15.913613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:15.913891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:15.914908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:15.917965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:15.953022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:15.981357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 844/1690] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:15 2022] POST /identity/v3/users => generated 304 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:15.984023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:16.000741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52490e8-e2d8-44d0-bfa5-1cb5512a3bc4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:16.036723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:16.039824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:16.040152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:16.040446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:16.042508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:16.049351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:16.070632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 847/1691] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:16.073421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:16.099164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec93818-8168-423b-9851-64519e7f6528 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:16.113095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:16.114003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:16.114461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:16.115106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:16.115944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:16.117774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ef2b9-f641-4ad2-aee7-cf5fc970291a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:16.148487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 845/1692] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:16.154961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:16.170437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f749792-746f-4d29-948f-45abd42c3335 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:16.193700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:16.194550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:16.196676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:16.197236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] PATH_INFO: `/v3/projects/ae7f491045b74c619f62b8a6770d1734/users/896a2ec899634645b473cf03d6610b50/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:16.251489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7f491045b74c619f62b8a6770d1734, user_id=896a2ec899634645b473cf03d6610b50, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:16.261754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:16.307294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 848/1693] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:16 2022] PUT /identity/v3/projects/ae7f491045b74c619f62b8a6770d1734/users/896a2ec899634645b473cf03d6610b50/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:16.308757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:16.309829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:16.309920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:16.368155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] MFA Rules not processed for user `896a2ec899634645b473cf03d6610b50`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:16.368155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:16.587979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 846/1694] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:16 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:16.589299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:16.774910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d59dac-f38f-4daa-842b-8a1c6c442979 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:16.788833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:16.789507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:16.789794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:16.790112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:16.790756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:16.801925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:16.877217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 849/1695] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:16 2022] POST /identity/v3/projects => generated 409 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:16.881304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:16.901152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476cd1-609b-4fa3-a658-a75a4c7aa5fc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.091297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.092212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.092689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.093169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.093995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.109216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.153993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 847/1696] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:16 2022] POST /identity/v3/users => generated 311 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:17.163607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.180575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad1bb-10cc-42cf-b549-0145651f8f64 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.199993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.200639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.200913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.201182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.202006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.205710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.242687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 850/1697] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:17.243917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.269980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a142-d6f1-4aec-adc7-d2b0bfcaba12 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.292956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.293847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.294127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.294400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] PATH_INFO: `/v3/projects/517a2dc877b04997b2de58562681aba2/users/581bcd80c0b94755b3f21a0439831221/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.331587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517a2dc877b04997b2de58562681aba2, user_id=581bcd80c0b94755b3f21a0439831221, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.334263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.349170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.351164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:17.368213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 848/1698] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:17 2022] PUT /identity/v3/projects/517a2dc877b04997b2de58562681aba2/users/581bcd80c0b94755b3f21a0439831221/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:17.380370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.515548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:17.522386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ba4ed-6b5c-42f8-9d05-7887522b893d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.548026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:17.564540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbfd8a-2217-491a-99e2-8da7e7336779 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.576592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.577667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.578117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.578570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.579463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.583601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.625134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 849/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:17.629201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.661073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db47eb-57cc-4259-bd8b-5b6e540870f9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:17.669047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.670069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.670350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.670624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.672484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:17.673797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:17.674440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.674775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.675076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] PATH_INFO: `/v3/projects/517a2dc877b04997b2de58562681aba2/users/581bcd80c0b94755b3f21a0439831221/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.680311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=517a2dc877b04997b2de58562681aba2, user_id=581bcd80c0b94755b3f21a0439831221, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.687552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde97d8-dd82-4fdb-bab5-54c143515ecd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.714698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 850/1700] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:17 2022] PUT /identity/v3/projects/517a2dc877b04997b2de58562681aba2/users/581bcd80c0b94755b3f21a0439831221/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:17.728371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:17.728654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:17.728915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:17.801525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] MFA Rules not processed for user `581bcd80c0b94755b3f21a0439831221`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:17.803304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:17.850620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:17.855332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:17.898117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 851/1701] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:17.973473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 851/1702] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:17 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:17.983640 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:17.985469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:18.016915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff19b4-a809-4a79-a4df-f091c90677a4 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:18.165110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:18.166251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:18.166868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:18.167368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:18.189869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:18.198468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:18.262476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 852/1703] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:17 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:18.483669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:18.484955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:18.728109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2d6d2-88bd-4323-83e9-577471a361b6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:18.747087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:18.758422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9477bb-5999-42ef-9a02-15385055fdf6 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:18.766830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:18.767521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:18.767797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:18.768070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] PATH_INFO: `/v3/projects/d2d3d0d44ab748d5bcdc09452916b6bc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:18.776453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:18.784888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:18.785084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:18.785331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:18.786070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d3d0d44ab748d5bcdc09452916b6bc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:18.791712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:18.994271 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d3d0d44ab748d5bcdc09452916b6bc', 'request_id': 'req-519abe5a-7e53-47c1-833e-54087e454ed5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:18.996305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:19.019633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d3d0d44ab748d5bcdc09452916b6bc', 'request_id': 'req-519abe5a-7e53-47c1-833e-54087e454ed5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:19.047447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d3d0d44ab748d5bcdc09452916b6bc', 'request_id': 'req-519abe5a-7e53-47c1-833e-54087e454ed5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:19.071059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:19.073625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69059461-f6a9-4094-a2bc-a1d19714fb43 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:19.104562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] The token cache is being invalidate because project d2d3d0d44ab748d5bcdc09452916b6bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:19.105362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:19.107075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-519abe5a-7e53-47c1-833e-54087e454ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:19.111004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 853/1704] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:18 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:19.121095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 852/1705] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:18 2022] DELETE /identity/v3/projects/d2d3d0d44ab748d5bcdc09452916b6bc => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:19.127904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafde71-681b-4926-9e59-5bf6b47cb51a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.128398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafde71-681b-4926-9e59-5bf6b47cb51a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.128815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafde71-681b-4926-9e59-5bf6b47cb51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.158193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:19.188846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2aafde71-681b-4926-9e59-5bf6b47cb51a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:19.190679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2aafde71-681b-4926-9e59-5bf6b47cb51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:19.312290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.312578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.312858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.400965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] MFA Rules not processed for user `478a0e3ac431440582d3dca1fe246ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:19.402220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:19.505393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 854/1706] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:19.507935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.510687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.512884 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.589983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:19.594843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:19.633096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 855/1707] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:19.635206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:19.643367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 853/1708] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:19.670339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:19.670858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a862062-914e-4525-a50f-7e7e3bad6b23 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:19.673537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:19.695784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:19.696654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.697308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.697866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.698603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:19.699532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:19.700895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:19.729580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 856/1709] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:19.736191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:19.777357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7360-1668-4ecf-a989-61990656aa42 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:19.793129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:19.797545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.799046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.799337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.807883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:19.816648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:19.872772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ba54-56bd-48a3-92c4-1051640fae64 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:19.882398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:19.883193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:19.883714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:19.883714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:19.904371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:19.905802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:19.929162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 854/1710] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:19.947826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 857/1711] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:19 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:19.948531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:19.967542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a202-2839-478f-b7b9-24455f5f4393 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:20.100617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:20.135561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:20.136597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.136888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.137219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.137861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:20.139979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:20.206926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 855/1712] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:19 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:20.209050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.223655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4447d4a-8b15-4b43-821e-cc4dd3c3fcd9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:20.234585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:20.235459 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.235908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.237061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.239179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:20.242862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:20.288019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.290043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 858/1713] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:20.307989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c08aa-f6cb-4182-b709-af4046c882b2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:20.319347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:20.320133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.321168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.322657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.324372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:20.326347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9e315-fe77-4ef3-b087-e75e1cb1223e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:20.345831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.346385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 856/1714] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:20.358568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78a86d-9156-4ee9-9726-b327f03614c9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:20.364167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:20.364868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.365247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.365782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] PATH_INFO: `/v3/projects/f489b66ff8544ca78a09ef9c13b76d28/users/a2e17627f6584681bf8b5f13d621e2a0/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.385324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f489b66ff8544ca78a09ef9c13b76d28, user_id=a2e17627f6584681bf8b5f13d621e2a0, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:20.389063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f03b7-4166-4139-b17a-8f3a0a9fcf8b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:20.421777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa231-9d28-4dd4-bc2c-fe67e64f47f9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.422170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa231-9d28-4dd4-bc2c-fe67e64f47f9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.422551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63faa231-9d28-4dd4-bc2c-fe67e64f47f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.454550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 859/1715] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:20 2022] PUT /identity/v3/projects/f489b66ff8544ca78a09ef9c13b76d28/users/a2e17627f6584681bf8b5f13d621e2a0/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:20.473632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68114-296b-4c03-95af-47a335cc468e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.474755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68114-296b-4c03-95af-47a335cc468e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.475234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe68114-296b-4c03-95af-47a335cc468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.504188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-63faa231-9d28-4dd4-bc2c-fe67e64f47f9 None None] MFA Rules not processed for user `f7491479bd88425f8234e0b1d0e11e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:20.505728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-63faa231-9d28-4dd4-bc2c-fe67e64f47f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:20.543010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-efe68114-296b-4c03-95af-47a335cc468e None None] MFA Rules not processed for user `a2e17627f6584681bf8b5f13d621e2a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:20.548710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-efe68114-296b-4c03-95af-47a335cc468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:20.659406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 857/1716] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:20.668071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.669542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.671317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.711090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 860/1717] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:20.718654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe68114-296b-4c03-95af-47a335cc468e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.720765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] MFA Rules not processed for user `f7491479bd88425f8234e0b1d0e11e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:20.723773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:20.742481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 858/1718] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:20.757211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.759142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:20.776965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:20.852945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe68114-296b-4c03-95af-47a335cc468e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:20.860339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:20.861189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:20.861576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:20.861949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:20.863583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:20.866226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:20.924363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 861/1719] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:20.926383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:20.939862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a6406-281c-4590-8341-dbbff2b91bf7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.037046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985660-f485-4a50-b682-1f10ee8d8626 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.046519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.047403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.048003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.048406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.049308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.050166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.054664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.088245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.090136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.090377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.090654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.125771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 862/1720] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:20 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:21.130091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.150301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8400-4fde-4b64-be60-891e2d383800 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.179352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.180662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.181399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.181907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.182672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.185820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.231463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 863/1721] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:21.237604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.268709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af32c45-c759-4c60-a97c-d9e00fc9ec06 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.301841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.301841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.301841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.301841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] PATH_INFO: `/v3/projects/3312b26681814de0a658ed5f3be296d4/users/6af724596e794024848dd409b7da031a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.302255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.315945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.366903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 859/1722] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:21.373821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3312b26681814de0a658ed5f3be296d4, user_id=6af724596e794024848dd409b7da031a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.379531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aae30-f0b3-4580-98fc-3190c3649eee admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.417797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 864/1723] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:21 2022] PUT /identity/v3/projects/3312b26681814de0a658ed5f3be296d4/users/6af724596e794024848dd409b7da031a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:21.420350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.447056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967537f9-8a51-435d-859c-6c94e9738369 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.657935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.658857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.659864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.660847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.675103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.684646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.705749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dfd2-6718-41ba-90f9-f7718d9b5221 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.706262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dfd2-6718-41ba-90f9-f7718d9b5221 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.706746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dfd2-6718-41ba-90f9-f7718d9b5221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.719691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 860/1724] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:21.726718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:21.759245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-caf8dfd2-6718-41ba-90f9-f7718d9b5221 None None] MFA Rules not processed for user `e18740d304014f8caa523b5d41500140`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.760531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-caf8dfd2-6718-41ba-90f9-f7718d9b5221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:21.771729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ad8a6-c629-420b-84a7-da5d4d6aa148 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:21.795208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:21.796417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.796923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.797421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] PATH_INFO: `/v3/projects/3312b26681814de0a658ed5f3be296d4/users/6af724596e794024848dd409b7da031a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.814367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3312b26681814de0a658ed5f3be296d4, user_id=6af724596e794024848dd409b7da031a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:21.817616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:21.851143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b675-c386-47e3-8bab-ca44be0c6c4e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:21.866263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 865/1725] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:21.868007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.868290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.868571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.902595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 861/1726] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:21 2022] PUT /identity/v3/projects/3312b26681814de0a658ed5f3be296d4/users/6af724596e794024848dd409b7da031a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:21.905744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:21.906892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:21.907961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:21.942811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] MFA Rules not processed for user `e18740d304014f8caa523b5d41500140`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.946765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:21.983942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] MFA Rules not processed for user `6af724596e794024848dd409b7da031a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:21.986536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.124345 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 866/1727] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:22.148043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.169027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.188187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 862/1728] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:22.196953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.199118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.328091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a759fe-1d1f-4534-9ae6-c7402a87cdfe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.337110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.337858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.338183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.338629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.354513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ddbbe-e7c0-4cb7-a2e1-f08882b9d9d6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.356904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.360087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.371015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.372436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.372718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.372996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.385130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 863/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:22.386200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.389183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.395966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.397592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.414722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 867/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:22.552745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:22.571259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f43d5-f03c-4011-9598-e9f71d4c3022 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.585314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.586220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.586698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.587186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.656993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd9ade-364d-4c8b-ad79-d9ad75a16c30 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:22.678711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:22.679477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.679752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.680034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] PATH_INFO: `/v3/users/125f71f3e47043ae905948146b485088` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.681736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=125f71f3e47043ae905948146b485088)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.684607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.804035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:22.809155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b52cf-4c01-422e-980f-c3570c3cc1c9 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:22.817667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125f71f3e47043ae905948146b485088'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:22.834403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125f71f3e47043ae905948146b485088'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:22.834791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 868/1731] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:22.835565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125f71f3e47043ae905948146b485088'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:22.836362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.836793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.837073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:22.865550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] MFA Rules not processed for user `cb11c04acb4f40cd9d09edf16b11abb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:22.866776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:22.870263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125f71f3e47043ae905948146b485088'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:22.897173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:22.901846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9eff7a7d-b801-43c1-820f-24cde84cb9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125f71f3e47043ae905948146b485088'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:22.931724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 864/1732] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:22 2022] DELETE /identity/v3/users/125f71f3e47043ae905948146b485088 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:22.935486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e363-ddee-4906-9356-78ca902a1686 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:22.939282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e363-ddee-4906-9356-78ca902a1686 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:22.939680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5e363-ddee-4906-9356-78ca902a1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.003844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-25d5e363-ddee-4906-9356-78ca902a1686 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:23.005107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25d5e363-ddee-4906-9356-78ca902a1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:23.012812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 869/1733] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:23.046154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:23.195012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 865/1734] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:23.209244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:23.210812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:23.509413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ef5f-5844-429f-92ca-f759951225bd None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:23.713324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:23.713953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:23.714379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:23.714662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:23.735745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:23.737381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf8273-81d5-4501-9a80-efa7f19e1b95 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:23.749645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 870/1735] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:24.102845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.103371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.103641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.105622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 866/1736] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 9 17:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:24.114862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.115695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.115695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.233823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:24.233997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:24.275403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 871/1737] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:24.374397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:24.379588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:24.437934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d545d0-4471-4319-b196-7eeaccf79f8f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:24.450630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:24.451534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.456029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.456290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] PATH_INFO: `/v3/projects/b41b4d5f87fd4a48b4c8e8d1c40d1e50` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.456431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:24.459094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:24.459225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41b4d5f87fd4a48b4c8e8d1c40d1e50)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:24.463752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:24.480308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5602b-f16a-4584-9423-acce48c714fb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:24.502647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:24.503333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.503725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.504007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.523230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:24.524930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4f988-e033-439f-8dce-c652e70a05de service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:24.541656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 872/1738] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:24.552311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41b4d5f87fd4a48b4c8e8d1c40d1e50', 'request_id': 'req-14b10110-0a09-4195-b9f2-9f888bd74e6b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.581256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41b4d5f87fd4a48b4c8e8d1c40d1e50', 'request_id': 'req-14b10110-0a09-4195-b9f2-9f888bd74e6b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.594772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7857218-badd-4f16-a9e8-589ffc64fdc7 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.595358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7857218-badd-4f16-a9e8-589ffc64fdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.595794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7857218-badd-4f16-a9e8-589ffc64fdc7 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.597620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 873/1739] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 9 17:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 17:56:24.602748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41b4d5f87fd4a48b4c8e8d1c40d1e50', 'request_id': 'req-14b10110-0a09-4195-b9f2-9f888bd74e6b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.604229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18572-942b-440f-a94e-65677b511497 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.604679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18572-942b-440f-a94e-65677b511497 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.605261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18572-942b-440f-a94e-65677b511497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.632410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] The token cache is being invalidate because project b41b4d5f87fd4a48b4c8e8d1c40d1e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:24.632717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.635476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.641862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 867/1740] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:24 2022] DELETE /identity/v3/projects/b41b4d5f87fd4a48b4c8e8d1c40d1e50 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:24.646037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:24.648403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:24.648682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cab18572-942b-440f-a94e-65677b511497 None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:24.650182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cab18572-942b-440f-a94e-65677b511497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:24.753104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:24.770655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 874/1741] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:56:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 09 17:56:24.791329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b10110-0a09-4195-b9f2-9f888bd74e6b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:24.797060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:24.797713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:24.797992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:24.798267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] PATH_INFO: `/v3/users/25c4209c251a4e7c9cd75cd51cefa1a3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:24.799808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c4209c251a4e7c9cd75cd51cefa1a3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:24.803469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:24.890046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:24.894609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c4209c251a4e7c9cd75cd51cefa1a3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.924729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c4209c251a4e7c9cd75cd51cefa1a3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.925724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c4209c251a4e7c9cd75cd51cefa1a3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.949563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c4209c251a4e7c9cd75cd51cefa1a3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.964657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-60d86a22-e9cf-4057-92a9-441b60d39619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c4209c251a4e7c9cd75cd51cefa1a3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:24.976211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 868/1742] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:24 2022] DELETE /identity/v3/users/25c4209c251a4e7c9cd75cd51cefa1a3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:25.054425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab18572-942b-440f-a94e-65677b511497 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:25.068010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cab18572-942b-440f-a94e-65677b511497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:25.085295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c9a2-02ed-4ab9-8f22-f8570c4e8930 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:25.085576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c9a2-02ed-4ab9-8f22-f8570c4e8930 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:25.085901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3c9a2-02ed-4ab9-8f22-f8570c4e8930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:25.118961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab18572-942b-440f-a94e-65677b511497 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:25.129572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c5f3c9a2-02ed-4ab9-8f22-f8570c4e8930 None None] MFA Rules not processed for user `18bf10b172e7450a9b618683643ed48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:25.131818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c5f3c9a2-02ed-4ab9-8f22-f8570c4e8930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:25.276677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 869/1743] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:25.343700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:25.344556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:25.344899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:25.345181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:25.349561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:25.398921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:25.407796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec58ba2-2e46-4c36-b9af-986cd7d81196 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:25.443545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 875/1744] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:25.953073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fb8c6-e1e1-4982-8649-0c6df53abbc1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:25.953672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fb8c6-e1e1-4982-8649-0c6df53abbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:25.954033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fb8c6-e1e1-4982-8649-0c6df53abbc1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:25.955772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 870/1745] 10.222.0.52 () {54 vars in 961 bytes} [Tue Aug 9 17:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:25.964851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cacf-9d8b-45f6-a792-53b9804ce178 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:25.965358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cacf-9d8b-45f6-a792-53b9804ce178 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:25.965725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515cacf-9d8b-45f6-a792-53b9804ce178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.001820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e515cacf-9d8b-45f6-a792-53b9804ce178 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:26.005959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e515cacf-9d8b-45f6-a792-53b9804ce178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.196318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 876/1746] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Aug 9 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:26.205735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fb8c6-e1e1-4982-8649-0c6df53abbc1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.223110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fb8c6-e1e1-4982-8649-0c6df53abbc1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:26.229396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:26.230283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.230733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.231145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.233316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.250580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:26.253221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:26.281631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 871/1747] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:26.343312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.343682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.344381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.384177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] MFA Rules not processed for user `478a0e3ac431440582d3dca1fe246ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:26.385419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.555482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.557817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.585972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 877/1748] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:26.739796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143803a-f431-450d-a6cb-9fc5bccdd9fd service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:26.760703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:26.761438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:26.763951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:26.763951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] PATH_INFO: `/v3/projects/79d9b2bb72044816aa6e80b443a76d1d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:26.763951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d9b2bb72044816aa6e80b443a76d1d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:26.765796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:26.890125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:26.911168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d9b2bb72044816aa6e80b443a76d1d', 'request_id': 'req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:26.920267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.922109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:26.936518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d9b2bb72044816aa6e80b443a76d1d', 'request_id': 'req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:26.946544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d9b2bb72044816aa6e80b443a76d1d', 'request_id': 'req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:26.980727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] The token cache is being invalidate because project 79d9b2bb72044816aa6e80b443a76d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:26.981362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:26.982510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:26.984562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 872/1749] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:26 2022] DELETE /identity/v3/projects/79d9b2bb72044816aa6e80b443a76d1d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:26.988529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:26.993478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:27.124870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:27.239324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ad7cd-e219-4eb8-9b36-75feb36d5aa9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.257269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c2c4-8e28-470c-9d99-e3d4c8686ce9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:27.278339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.279039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.279336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.279593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] PATH_INFO: `/v3/users/3b34ec4cd3234daf802ef82ad3f1802f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.283296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b34ec4cd3234daf802ef82ad3f1802f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.301758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.452023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b34ec4cd3234daf802ef82ad3f1802f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:27.467890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:27.468744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:27.469364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:27.470431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:27.471559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b34ec4cd3234daf802ef82ad3f1802f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:27.472796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b34ec4cd3234daf802ef82ad3f1802f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:27.479192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:27.512999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b34ec4cd3234daf802ef82ad3f1802f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:27.551759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2341f4b6-d90b-4ae0-b418-30db6b8e2c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b34ec4cd3234daf802ef82ad3f1802f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:27.566773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 873/1750] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:26 2022] DELETE /identity/v3/users/3b34ec4cd3234daf802ef82ad3f1802f => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:27.686966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:27.699431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:27.735682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 878/1751] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:28.090284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa089b3-8e65-4841-96ee-085b8e318fe0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:28.090284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa089b3-8e65-4841-96ee-085b8e318fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:28.090501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa089b3-8e65-4841-96ee-085b8e318fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:28.137234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7fa089b3-8e65-4841-96ee-085b8e318fe0 None None] MFA Rules not processed for user `f7491479bd88425f8234e0b1d0e11e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:28.139211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7fa089b3-8e65-4841-96ee-085b8e318fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:28.234036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:28.250652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 874/1752] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:29.054448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.073266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f4fd-63cf-4ec3-9e73-058f22c54d01 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.182449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.183309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.183688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.184113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] PATH_INFO: `/v3/projects/fd6c9f530a9c4423a8521e4971095f88` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.185495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6c9f530a9c4423a8521e4971095f88)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.187975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.209210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:29.254283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6c9f530a9c4423a8521e4971095f88', 'request_id': 'req-7580cb5c-2035-4b81-ac8f-63cde59e9b91'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:29.262668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6c9f530a9c4423a8521e4971095f88', 'request_id': 'req-7580cb5c-2035-4b81-ac8f-63cde59e9b91'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:29.271673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6c9f530a9c4423a8521e4971095f88', 'request_id': 'req-7580cb5c-2035-4b81-ac8f-63cde59e9b91'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:29.290158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] The token cache is being invalidate because project fd6c9f530a9c4423a8521e4971095f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:29.290636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:29.291478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7580cb5c-2035-4b81-ac8f-63cde59e9b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:29.294738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 879/1753] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:29 2022] DELETE /identity/v3/projects/fd6c9f530a9c4423a8521e4971095f88 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:29.301074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.301538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.301642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.346943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:29.348303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:29.442810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 875/1754] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:29.451815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cede-a281-433b-8e14-f985e022462a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.452156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cede-a281-433b-8e14-f985e022462a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.452429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40cede-a281-433b-8e14-f985e022462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.514407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4c40cede-a281-433b-8e14-f985e022462a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:29.515992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4c40cede-a281-433b-8e14-f985e022462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:29.543825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 880/1755] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:29.545553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.558638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7595ac9-bcda-4dba-a7d9-8dccbf46969b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.581801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.582431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.582709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.583223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.584078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:29.584541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.587202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.615804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 876/1756] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:29.620215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40cede-a281-433b-8e14-f985e022462a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.636629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40cede-a281-433b-8e14-f985e022462a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.653377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.654620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.654927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.655218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.660021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.664262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.740732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 881/1757] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:29 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:29.745865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:29.778887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c3230-96ee-4fd1-bd82-6dfdf2d6c1e5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:29.929414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:29.930119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:29.930412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:29.933946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:29.935049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:29.943050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:29.995512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 877/1758] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:29 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:30.000471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.020830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387e295-d2cc-4699-bc70-2c93417b01e4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.042797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.044921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.047399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.048199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.050010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.058062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.087519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 882/1759] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:30.092880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.104159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2535-2ef7-4bf4-ab82-20327ee3be2a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.110905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.117977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.118268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.118548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.119799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.122338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e0155-7a30-43fd-8d58-c11f6d3030c7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.160109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 878/1760] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:30.161330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.190766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8597f-cd8f-4723-a1f3-5c6726d7ea19 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.212185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.212971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.213315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.213518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] PATH_INFO: `/v3/projects/efc9b3896f174d739be9ff7eeadb0146/users/38023e6a40d642568377485935d1fdaf/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.264995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc9b3896f174d739be9ff7eeadb0146, user_id=38023e6a40d642568377485935d1fdaf, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.270227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f4698-5bbc-4719-9ae9-4d073693256f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.296737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 883/1761] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:30 2022] PUT /identity/v3/projects/efc9b3896f174d739be9ff7eeadb0146/users/38023e6a40d642568377485935d1fdaf/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:30.312980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.314242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.314691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.316831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.317289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.317841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.364697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] MFA Rules not processed for user `18bf10b172e7450a9b618683643ed48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:30.368123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:30.377861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] MFA Rules not processed for user `38023e6a40d642568377485935d1fdaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:30.380128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:30.486587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:30.532203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 884/1762] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:30.532916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 879/1763] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:30.538985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.650046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:30.701908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ff908-26bd-42ec-b77d-318f664fbfd4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.709418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.710216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.710641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.711144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.712275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.714662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.771084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 885/1764] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/projects => generated 383 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:30.775258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.788038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a05e-15db-4aa7-a6ef-c858fd71f4c0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.865702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.866579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.867211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.867626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.868440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.870625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.899918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 880/1765] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:30 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:30.904800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.915680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ad9ba-36f2-4f50-a843-78043ee3533b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.922680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.923390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.923693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.924013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:30.924686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:30.927891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:30.963309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:30.967192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 886/1766] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:30.983475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b1991-fd81-497a-a7f3-863419b2e3cc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:30.992034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:30.996139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:30.996492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:30.996773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] PATH_INFO: `/v3/projects/340e43506af3407aab7873a90e582e35/users/21e56a3976cd4b249c8b039384ed48f2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.024905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e43506af3407aab7873a90e582e35, user_id=21e56a3976cd4b249c8b039384ed48f2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.028313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.055478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 881/1767] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:30 2022] PUT /identity/v3/projects/340e43506af3407aab7873a90e582e35/users/21e56a3976cd4b249c8b039384ed48f2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:31.059565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.077849 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f4c59-7984-47ec-bcf0-b4c08237c92d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.195595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.196470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.196959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.197379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.198257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.200399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96bf50-c53c-406e-8874-5359b72562c9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.212744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 887/1768] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:31.217890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.231660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a3a4-d8a9-403f-ad44-2271744ab3e6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.237380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.238664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.239373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.239837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] PATH_INFO: `/v3/projects/340e43506af3407aab7873a90e582e35/users/21e56a3976cd4b249c8b039384ed48f2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.243461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340e43506af3407aab7873a90e582e35, user_id=21e56a3976cd4b249c8b039384ed48f2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.246936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.261272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 882/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:31 2022] PUT /identity/v3/projects/340e43506af3407aab7873a90e582e35/users/21e56a3976cd4b249c8b039384ed48f2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:31.271502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446167f-acbd-4066-883f-49449427e562 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.271826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446167f-acbd-4066-883f-49449427e562 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.272240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446167f-acbd-4066-883f-49449427e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.323515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4446167f-acbd-4066-883f-49449427e562 None None] MFA Rules not processed for user `21e56a3976cd4b249c8b039384ed48f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:31.325141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4446167f-acbd-4066-883f-49449427e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:31.442120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 888/1770] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:31.454010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:31.456470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:31.619611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5c9-4810-4d74-909a-7891dc0fa282 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:31.642058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:31.642707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:31.642996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:31.643287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:31.686937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:31.688558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:31.711688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 883/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:32.001571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446167f-acbd-4066-883f-49449427e562 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.017792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446167f-acbd-4066-883f-49449427e562 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.029900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.030788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.031425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.031710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.178995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.204790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.205088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.206274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667de48-9e53-433a-b0d2-33f0261f0dd8 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.206966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:32.230224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 889/1772] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:32.358363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:32.441774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edb25-4d8d-436b-a03f-27a106b3c7d1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:32.457441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:32.458303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.458621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.458895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] PATH_INFO: `/v3/users/896a2ec899634645b473cf03d6610b50` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.460529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=896a2ec899634645b473cf03d6610b50)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:32.469672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:32.594094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896a2ec899634645b473cf03d6610b50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:32.606903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896a2ec899634645b473cf03d6610b50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:32.607585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896a2ec899634645b473cf03d6610b50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:32.625067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896a2ec899634645b473cf03d6610b50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:32.636238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896a2ec899634645b473cf03d6610b50'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:32.649126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 884/1773] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:32 2022] DELETE /identity/v3/users/896a2ec899634645b473cf03d6610b50 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:32.654116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:32.654592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:32.654950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:32.697622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:32.701221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:32.832174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 890/1774] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:32.851841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:32.853580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:33.033463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5b791-33d2-45d6-a0f5-af8e3484f9b2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.037663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.038441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.039104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.039698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.098303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.099934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.112225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 885/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:33.569846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.572075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:33.596384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8886413-798b-4404-bdc5-301d9d40bfba None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.607607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.610247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.611919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.612761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] PATH_INFO: `/v3/projects/ae7f491045b74c619f62b8a6770d1734` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.615222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7f491045b74c619f62b8a6770d1734)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.620648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.748903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7f491045b74c619f62b8a6770d1734', 'request_id': 'req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:33.762788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7f491045b74c619f62b8a6770d1734', 'request_id': 'req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:33.780084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7f491045b74c619f62b8a6770d1734', 'request_id': 'req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:33.805943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] The token cache is being invalidate because project ae7f491045b74c619f62b8a6770d1734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:33.807329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:33.810008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-900e1a71-4bde-4ba2-b3c5-6cba3021fbf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:33.813742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 891/1776] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:33 2022] DELETE /identity/v3/projects/ae7f491045b74c619f62b8a6770d1734 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:33.819359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:33.834957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b541d7c-c071-48a0-9e05-0a29cd708e5d service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:33.971694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:33.972761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:33.973959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:33.974449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] PATH_INFO: `/v3/users/581bcd80c0b94755b3f21a0439831221` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:33.975924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581bcd80c0b94755b3f21a0439831221)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:33.978554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:33.992789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.092278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581bcd80c0b94755b3f21a0439831221'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:34.124290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581bcd80c0b94755b3f21a0439831221'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:34.125217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581bcd80c0b94755b3f21a0439831221'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:34.161881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581bcd80c0b94755b3f21a0439831221'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:34.177901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581bcd80c0b94755b3f21a0439831221'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:34.187525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 886/1777] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:33 2022] DELETE /identity/v3/users/581bcd80c0b94755b3f21a0439831221 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:34.282075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.282378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.282812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.397902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:34.399317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:34.449990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:34.499834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 892/1778] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:56:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:34.635018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:34.635018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:34.925390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18653d-0b21-4dde-992b-db6ee01ffb78 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:34.932194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:34.932835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:34.933107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:34.933548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:34.952661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:34.954527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35615392-d5f6-4fdd-8657-ac40524058b7 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:34.974168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 887/1779] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:56:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:56:35.619361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:35.628392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:35.798512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fe0cb-ecbd-4833-bd67-2c718b93faf2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:35.813163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:35.813868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:35.814156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:35.814432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] PATH_INFO: `/v3/users/38023e6a40d642568377485935d1fdaf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:35.816281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38023e6a40d642568377485935d1fdaf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:35.820205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:35.946253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38023e6a40d642568377485935d1fdaf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:35.971269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38023e6a40d642568377485935d1fdaf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:35.972663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38023e6a40d642568377485935d1fdaf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.002782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38023e6a40d642568377485935d1fdaf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.053756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38023e6a40d642568377485935d1fdaf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.083255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 893/1780] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:35 2022] DELETE /identity/v3/users/38023e6a40d642568377485935d1fdaf => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:36.092778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.092778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.093331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.134857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.136342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.274966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.305224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 888/1781] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:36.323489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.325186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.447866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52157792-d8b3-488d-a261-28bf1a0bb9f6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.456120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.456947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.457367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.457774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.488166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.489879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.494781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:36.513166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.514608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.541332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 894/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:36.542231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:36.544280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.544451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf3815-3dcb-4ffa-8d3d-cfa938a350d3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.577248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.579086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.580287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.580420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a044a94-e38d-4221-aa24-2e62b39eef51 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:36.582500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] PATH_INFO: `/v3/projects/517a2dc877b04997b2de58562681aba2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.604483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517a2dc877b04997b2de58562681aba2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.607745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:36.608415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.608707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.608995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] PATH_INFO: `/v3/users/a2e17627f6584681bf8b5f13d621e2a0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.611388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e17627f6584681bf8b5f13d621e2a0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:36.613338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.623703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:36.773476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e17627f6584681bf8b5f13d621e2a0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.802309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517a2dc877b04997b2de58562681aba2', 'request_id': 'req-5870951e-ca42-4873-a5bf-3c3158d92e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.807852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e17627f6584681bf8b5f13d621e2a0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.810099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e17627f6584681bf8b5f13d621e2a0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.823594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517a2dc877b04997b2de58562681aba2', 'request_id': 'req-5870951e-ca42-4873-a5bf-3c3158d92e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.836689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e17627f6584681bf8b5f13d621e2a0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.857525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517a2dc877b04997b2de58562681aba2', 'request_id': 'req-5870951e-ca42-4873-a5bf-3c3158d92e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.889152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] The token cache is being invalidate because project 517a2dc877b04997b2de58562681aba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:36.889581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.895171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-04981641-8946-4137-ade5-a57663ecd06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e17627f6584681bf8b5f13d621e2a0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.905001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5870951e-ca42-4873-a5bf-3c3158d92e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:36.911550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 889/1783] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:36 2022] DELETE /identity/v3/projects/517a2dc877b04997b2de58562681aba2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:36.913269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3a1a6-2f54-48b9-8621-6a3deab87e89 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.915031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3a1a6-2f54-48b9-8621-6a3deab87e89 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.915388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3a1a6-2f54-48b9-8621-6a3deab87e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.919423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 895/1784] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:36 2022] DELETE /identity/v3/users/a2e17627f6584681bf8b5f13d621e2a0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:36.924243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:36.924521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:36.925070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:36.968501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-58d3a1a6-2f54-48b9-8621-6a3deab87e89 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.970124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-58d3a1a6-2f54-48b9-8621-6a3deab87e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:36.976571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:36.978277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:37.149065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 890/1785] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:37.160347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.160347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.166333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 896/1786] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:37.166545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.206724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:37.208217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:37.235806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 891/1787] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:37.239844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.256535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dd9c18-e726-4951-a63a-dac2da402918 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.275864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.276768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.277045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.277311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.278542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:37.279118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.280699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790af8b-0299-42f5-b5cc-06266be810ca admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.296031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.297540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:37.338358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 897/1788] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:37.348759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.359373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.360009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.486160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:37.493415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:37.505980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ff8fc-0e94-42a4-8d5b-af75e1de4020 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.508078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 898/1789] 10.222.0.40 () {58 vars in 1015 bytes} [Tue Aug 9 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:37.511001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.523329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.523966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.524244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.524509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.532549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c441ecb-f712-4d94-9d3b-2436155b0bcc None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.549134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.556193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.556594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.557021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.557835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.560516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.570325 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.573567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.605502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 892/1790] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:37.617180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.631985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 899/1791] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:37 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:37.633337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.651520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636286d1-49cf-4df9-86af-e250f9b1e8e4 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.688346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.744870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:37.833384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9241a-6506-4791-ae76-f57650ecf2b7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.840679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.841590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.841590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.841905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.843046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.845847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.855524 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.856212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.856497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.860659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.893175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 900/1792] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:37 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:37.896045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:37.900002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.901468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.930411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 893/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:37.938591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989948-e3be-42b2-a009-b619745ccfc7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:37.948393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:37.948393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:37.948393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:37.948393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:37.948802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:37.950591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:37.988690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 901/1794] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:37.993042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.020201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c400ef5-ac69-432c-b08c-edd230f89144 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.035151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.036306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.036938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.037410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.038328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.040194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f10b71-6ed4-473d-9901-e3b4c2966b93 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.063481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 894/1795] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:38.067381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.082455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633940e2-03dd-4f84-a5c3-f4722d1b4e1f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.087262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.088072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.088321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.088605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] PATH_INFO: `/v3/projects/f0b514c56ad146499d3bbd78117e76ef/users/e72c69e3e9874295b2586e3f2c7c23ac/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.133995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b514c56ad146499d3bbd78117e76ef, user_id=e72c69e3e9874295b2586e3f2c7c23ac, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.140674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.165382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 902/1796] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:38 2022] PUT /identity/v3/projects/f0b514c56ad146499d3bbd78117e76ef/users/e72c69e3e9874295b2586e3f2c7c23ac/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:38.178871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.180119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.180563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.226063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] MFA Rules not processed for user `e72c69e3e9874295b2586e3f2c7c23ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:38.227633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:38.324298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 895/1797] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:38.326087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.506872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd17d7c-a2eb-4122-a06c-befdd6132606 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.512709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.513763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.514392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.514867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.541946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.550868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.675719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 903/1798] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:38 2022] POST /identity/v3/projects => generated 373 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:38.680165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.703321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0342085-f16a-4071-93e2-fd3fd4a98517 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.870421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.872340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.872988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.873469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.874541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.882072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.933538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.946097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 896/1799] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:38 2022] POST /identity/v3/users => generated 287 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:38.949044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089e1ef-4648-4280-a187-41f001e0074f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:38.955022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:38.955730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:38.956011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:38.956293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:38.957215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:38.959406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:38.978525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 904/1800] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:38.983425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:38.996033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e495-fc8e-4493-9324-81419f7e3dfe admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.003399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.004026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.004787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.005069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.005749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:39.007719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d051c-31c3-4c48-a0a1-1758525ac902 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:39.030895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.043341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 897/1801] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:39.044943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eceea4-7340-4275-a09b-95cb4ad14a21 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.051881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.052535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.052795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.053077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] PATH_INFO: `/v3/projects/11974f4e6a9d4d0992815fdbf2788633/users/8ca76cc1cf824bc4931fab6e2953c808/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.075684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11974f4e6a9d4d0992815fdbf2788633, user_id=8ca76cc1cf824bc4931fab6e2953c808, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:39.078848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:39.111706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.111915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 905/1802] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:39 2022] PUT /identity/v3/projects/11974f4e6a9d4d0992815fdbf2788633/users/8ca76cc1cf824bc4931fab6e2953c808/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:39.112238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.112667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.244269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] MFA Rules not processed for user `8ca76cc1cf824bc4931fab6e2953c808`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:39.248275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:39.258343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:39.457631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.458594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 898/1803] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:39.540200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:39.651673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b002744-a584-45ec-ba17-1dcd852a9082 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:39.673796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:39.675114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:39.675613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:39.677183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:39.678036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:39.696122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:39.770073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 906/1804] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:39.774115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:39.790702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6e9a9-ddbe-4925-bdd3-78d1ff8061d1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.007038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.007694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.007976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.008249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.008900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.017813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.092440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 899/1805] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:39 2022] POST /identity/v3/users => generated 295 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:40.095974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.111609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca889ba-d7bc-49c0-8c4f-67c2df7dc6b3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.122788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.123544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.123828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.124312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.124984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.127695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.194472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 907/1806] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:40.198576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.232232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1154b-0dfc-45ba-9b59-81a6941d6c8c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.247858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.252646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.253382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.255247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] PATH_INFO: `/v3/projects/bbabc9acc01f44fbbcf484be75f74a48/users/441b9ee75dba4a2f87475d6687a27940/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.303080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbabc9acc01f44fbbcf484be75f74a48, user_id=441b9ee75dba4a2f87475d6687a27940, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.312392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.346622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 900/1807] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:40 2022] PUT /identity/v3/projects/bbabc9acc01f44fbbcf484be75f74a48/users/441b9ee75dba4a2f87475d6687a27940/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:40.350607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.370388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e73d5-0f38-458b-8f26-0bcd47c37b4c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.494235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.495077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.495421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.495760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.497366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.499169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.507251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.509691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.531893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 908/1808] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:40.533567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.553414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c7585-00a3-4fa6-8232-5c7fdd8f21ff admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.553584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ab16-b2e9-4da4-968f-dd4b9191e48a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:40.565871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.566556 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.566830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.567128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] PATH_INFO: `/v3/projects/bbabc9acc01f44fbbcf484be75f74a48/users/441b9ee75dba4a2f87475d6687a27940/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.569481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbabc9acc01f44fbbcf484be75f74a48, user_id=441b9ee75dba4a2f87475d6687a27940, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.572694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c7499-4604-4ee1-bb27-00e51b9595d3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.573056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:40.575914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.575914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.576117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] PATH_INFO: `/v3/projects/efc9b3896f174d739be9ff7eeadb0146` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.586284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc9b3896f174d739be9ff7eeadb0146)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:40.594264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:40.612346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 909/1809] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:40 2022] PUT /identity/v3/projects/bbabc9acc01f44fbbcf484be75f74a48/users/441b9ee75dba4a2f87475d6687a27940/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:40.615314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:40.615314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:40.615464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:40.663014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] MFA Rules not processed for user `441b9ee75dba4a2f87475d6687a27940`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:40.664234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.741745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc9b3896f174d739be9ff7eeadb0146', 'request_id': 'req-201d0b5f-87c3-4e3c-a285-123a21948cdc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:40.760292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc9b3896f174d739be9ff7eeadb0146', 'request_id': 'req-201d0b5f-87c3-4e3c-a285-123a21948cdc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:40.782681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc9b3896f174d739be9ff7eeadb0146', 'request_id': 'req-201d0b5f-87c3-4e3c-a285-123a21948cdc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:40.800146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:40.815018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 910/1810] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:40.829964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.834846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.846404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] The token cache is being invalidate because project efc9b3896f174d739be9ff7eeadb0146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:40.846827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:40.847472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:40.849444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 901/1811] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:40 2022] DELETE /identity/v3/projects/efc9b3896f174d739be9ff7eeadb0146 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:40.856621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:40.859447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:40.894524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.029175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d0b5f-87c3-4e3c-a285-123a21948cdc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.037661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e6297-6c36-4a9c-af1a-fae5d87d7a2e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.039704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.040517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.040853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.041233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] PATH_INFO: `/v3/users/21e56a3976cd4b249c8b039384ed48f2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.043305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e56a3976cd4b249c8b039384ed48f2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.050847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.208825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e56a3976cd4b249c8b039384ed48f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.223031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.224168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.224573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.224906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.228711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e56a3976cd4b249c8b039384ed48f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.229427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:41.231208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e56a3976cd4b249c8b039384ed48f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.263759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e56a3976cd4b249c8b039384ed48f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.281384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e56a3976cd4b249c8b039384ed48f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.300098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 902/1812] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:40 2022] DELETE /identity/v3/users/21e56a3976cd4b249c8b039384ed48f2 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:41.301009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.305663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:41.389844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.393987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.415531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 911/1813] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:41.439498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5411f0d-1951-4fb3-a27a-f213bc3803fb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.445168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.445823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.446151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.446427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] PATH_INFO: `/v3/projects/f489b66ff8544ca78a09ef9c13b76d28` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.447823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f489b66ff8544ca78a09ef9c13b76d28)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.451572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:41.473641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:41.546484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f489b66ff8544ca78a09ef9c13b76d28', 'request_id': 'req-eac58825-24b1-4068-95df-35da84d84603'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.563885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f489b66ff8544ca78a09ef9c13b76d28', 'request_id': 'req-eac58825-24b1-4068-95df-35da84d84603'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.582948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f489b66ff8544ca78a09ef9c13b76d28', 'request_id': 'req-eac58825-24b1-4068-95df-35da84d84603'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.620726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] The token cache is being invalidate because project f489b66ff8544ca78a09ef9c13b76d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:41.623097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.624577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eac58825-24b1-4068-95df-35da84d84603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:41.629134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 903/1814] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:41 2022] DELETE /identity/v3/projects/f489b66ff8544ca78a09ef9c13b76d28 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:41.633249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:41.634906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:41.659633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a86e6-bd9d-44b6-9801-cd742528c260 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:41.903953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:41.906955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:41.909341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:41.909874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] PATH_INFO: `/v3/users/6af724596e794024848dd409b7da031a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:41.916664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af724596e794024848dd409b7da031a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:41.925642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:42.020484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:42.081443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af724596e794024848dd409b7da031a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:42.135950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af724596e794024848dd409b7da031a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:42.153597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af724596e794024848dd409b7da031a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:42.188829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af724596e794024848dd409b7da031a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:42.217239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af724596e794024848dd409b7da031a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:42.292628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 912/1815] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:41 2022] DELETE /identity/v3/users/6af724596e794024848dd409b7da031a => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:43.602881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:43.603369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:43.603369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:43.604772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 904/1816] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:43.827021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.829231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:43.891660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:43.896790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.077881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a9a9d-6ae6-46df-a44f-db02d5a31bc9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.097657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.099402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.099839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.100258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.106514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.176353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad34ac-dbd6-4122-b652-61afa7abc4da None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.188559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.189163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.189431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.189708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] PATH_INFO: `/v3/projects/340e43506af3407aab7873a90e582e35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.194128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340e43506af3407aab7873a90e582e35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.203956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.309956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.315527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d263b7-7e00-4554-8304-9171c174983e service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.336653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340e43506af3407aab7873a90e582e35', 'request_id': 'req-86b68622-def3-41ce-8772-368c01db7e25'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:44.360485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 913/1817] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:44.365721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340e43506af3407aab7873a90e582e35', 'request_id': 'req-86b68622-def3-41ce-8772-368c01db7e25'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:44.366348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.367959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.369071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.408520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340e43506af3407aab7873a90e582e35', 'request_id': 'req-86b68622-def3-41ce-8772-368c01db7e25'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:44.458228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] The token cache is being invalidate because project 340e43506af3407aab7873a90e582e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:44.458536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:44.459333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-86b68622-def3-41ce-8772-368c01db7e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:44.461507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 905/1818] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:43 2022] DELETE /identity/v3/projects/340e43506af3407aab7873a90e582e35 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:44.467656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d95-b437-4994-81ab-83f6580ac84b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.468152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d95-b437-4994-81ab-83f6580ac84b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.469479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d95-b437-4994-81ab-83f6580ac84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.485213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:44.490497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.522528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-38546d95-b437-4994-81ab-83f6580ac84b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:44.529581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-38546d95-b437-4994-81ab-83f6580ac84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.595585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.642736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:44.663394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 906/1819] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:44.672228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.673113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.673702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.693963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 914/1820] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:44.714596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:44.716699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:44.737953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 907/1821] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:44.743016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.761492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96d90e-43bc-45c9-88ec-22add2b8a5ff None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.769401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.771109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.771840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.772423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.774002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:44.774477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.776354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.796858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 915/1822] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:44.801256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.817282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46073d22-4053-499b-9ade-ec7ef8295223 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:44.851903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:44.856316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:44.856682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:44.857065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:44.864638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:44.867232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:44.940194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 908/1823] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:44.944361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:44.966196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814810f5-8ff0-4f80-8106-5ae50a0df286 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.219579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.221642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.222953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.224152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.229091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.233312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.314773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 916/1824] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:44 2022] POST /identity/v3/users => generated 295 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:45.319117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.337670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e74ed-9ee1-49ea-a7c3-970c978bce2f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.356474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.357113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.357396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.357668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.358318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.359950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.382350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 909/1825] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:45.386261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.406436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4673b0f3-4aba-4321-b778-c969978bf79e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.427779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.429601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.429961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.430317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.431088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.432842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471ddd3-a607-4550-a9bf-244f81a4cd11 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.468502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 917/1826] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:45.472584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.484654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9720eb6-9973-4cb4-9e0c-6d478aa025e7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:45.499489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:45.500112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.500412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.501106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] PATH_INFO: `/v3/projects/f2ffe7f4d7e2494590b50933e4612493/users/b63c7934ee804cb199af250507cc2e1e/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.559702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ffe7f4d7e2494590b50933e4612493, user_id=b63c7934ee804cb199af250507cc2e1e, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:45.565633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:45.616149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 910/1827] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:45 2022] PUT /identity/v3/projects/f2ffe7f4d7e2494590b50933e4612493/users/b63c7934ee804cb199af250507cc2e1e/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:45.618630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:45.618926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:45.619317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:45.685076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] MFA Rules not processed for user `b63c7934ee804cb199af250507cc2e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:45.686890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:45.876872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 918/1828] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:45.881196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.939960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:45.943637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:46.035327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef998f34-aa25-42c1-bae7-d9964eb4586c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.052857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.053508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.053823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.054116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.054906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.057175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.114006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.128448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 911/1829] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:45 2022] POST /identity/v3/projects => generated 389 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:46.131456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.172914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca33ee-1a86-4b8e-a633-9c70ae19ed27 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.250916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.257406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a33f4-39a2-435f-977f-f2767366c927 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.260590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.261266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.261558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.262656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.266009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.266892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.267439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.267922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.268755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.270907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.274494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.276325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.293936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 919/1830] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:46.293936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.293936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:46.321563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 912/1831] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:46 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:46.326441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.353308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937640ed-95db-4a47-8b65-97b479dd8ad5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.366611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.367594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.367894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.368843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.369846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.373431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.398685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 913/1832] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:46.404072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.430574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ea034-420d-44e3-9cc6-7ca13f32b4d3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.447067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509e27c-23d7-4f1f-a7dd-1069872488d7 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.448232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.450550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.450864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.451267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] PATH_INFO: `/v3/projects/c761dc85044a485082565ced8b09fb16/users/15b549b6e5bb4eb9836ab60cff94e215/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.460306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.462152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.462340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.462613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.473331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:46.536055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c761dc85044a485082565ced8b09fb16, user_id=15b549b6e5bb4eb9836ab60cff94e215, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.538816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b0ee-2b5a-4cbc-bedf-b595aea9da94 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.577965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 914/1833] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:46 2022] PUT /identity/v3/projects/c761dc85044a485082565ced8b09fb16/users/15b549b6e5bb4eb9836ab60cff94e215/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:46.579775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625efa98-7555-4648-9e00-c68f3bdfdc38 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.580066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625efa98-7555-4648-9e00-c68f3bdfdc38 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.582627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625efa98-7555-4648-9e00-c68f3bdfdc38 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.584254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 915/1834] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:56:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:46.585282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625efa98-7555-4648-9e00-c68f3bdfdc38 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.707947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625efa98-7555-4648-9e00-c68f3bdfdc38 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.715379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.716215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.716560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.716907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.717768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.720398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.748771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.753705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63761b-56d3-4a61-a6ed-6a22bdb2dca0 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.754734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 916/1835] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:46.758643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:46.773348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 920/1836] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:46.800270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.800895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.801374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.821355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2becc9a-fc2a-46ed-8d69-cae392d34bc2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:46.839532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:46.840627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.847822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.848148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] PATH_INFO: `/v3/projects/c761dc85044a485082565ced8b09fb16/users/15b549b6e5bb4eb9836ab60cff94e215/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.855582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c761dc85044a485082565ced8b09fb16, user_id=15b549b6e5bb4eb9836ab60cff94e215, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:46.864194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82578e3-f31c-4abb-a351-1932159c6dc0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:46.866651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:46.869692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:46.892114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 917/1837] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:46 2022] PUT /identity/v3/projects/c761dc85044a485082565ced8b09fb16/users/15b549b6e5bb4eb9836ab60cff94e215/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:46.893887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:46.894213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:46.894443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:46.909423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:46.937043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] MFA Rules not processed for user `15b549b6e5bb4eb9836ab60cff94e215`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:46.938370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.040908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 921/1838] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:47.107635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 918/1839] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:47.132140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.132651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.134522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.172009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.238151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446c00f-1962-4709-8430-265eb44abd4c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.249759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.250709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.251129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.251885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] PATH_INFO: `/v3/projects/3312b26681814de0a658ed5f3be296d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.255331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3312b26681814de0a658ed5f3be296d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.260383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.294248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf7b5e-0c8d-468c-a287-24c39ea4038c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.309725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.310473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.310818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.311112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.330897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.332576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48ace4-fbff-4a1e-b3f3-007f0f141f6e service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.372753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 922/1840] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:47.378596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3312b26681814de0a658ed5f3be296d4', 'request_id': 'req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.412536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3312b26681814de0a658ed5f3be296d4', 'request_id': 'req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.485352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3312b26681814de0a658ed5f3be296d4', 'request_id': 'req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.525709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] The token cache is being invalidate because project 3312b26681814de0a658ed5f3be296d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:47.527027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.533739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.542544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 919/1841] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:47 2022] DELETE /identity/v3/projects/3312b26681814de0a658ed5f3be296d4 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:47.544158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e0fe-4e99-4a53-b626-d991f0a4a8bc None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.544873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e0fe-4e99-4a53-b626-d991f0a4a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.545553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e0fe-4e99-4a53-b626-d991f0a4a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.603244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4c73e0fe-4e99-4a53-b626-d991f0a4a8bc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:47.605980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4c73e0fe-4e99-4a53-b626-d991f0a4a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.615537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.617554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.714325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 923/1842] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:47.725229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.725507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.725780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.754547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:47.756058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.772928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 924/1843] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:47.775634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.791073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50be4-abab-42a2-9a28-15e876bdda59 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.792728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a3a7c-e1ab-4b6e-8e86-e5371d7c1bd0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.798875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.799613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.799963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.800303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.801190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.801377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:47.801992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.802146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.802272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.802538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] PATH_INFO: `/v3/users/b63c7934ee804cb199af250507cc2e1e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.804126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63c7934ee804cb199af250507cc2e1e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.804240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.806140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.829089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 925/1844] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:47.837078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.865041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72275f0b-425f-4096-b69b-ca5245b974af admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:47.879302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:47.880708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:47.880988 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:47.881256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:47.882105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:47.884287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:47.951009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63c7934ee804cb199af250507cc2e1e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.958070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:47.967154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63c7934ee804cb199af250507cc2e1e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.968411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63c7934ee804cb199af250507cc2e1e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.981442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 926/1845] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:47.987985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:47.991468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63c7934ee804cb199af250507cc2e1e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:47.996071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:47.999681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bad28639-768a-4117-a1a6-b721f2e39a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63c7934ee804cb199af250507cc2e1e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:48.010403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 920/1846] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:47 2022] DELETE /identity/v3/users/b63c7934ee804cb199af250507cc2e1e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:48.016270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.016604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.016879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.045194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.074338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:48.074338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.130507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369669a4-39be-4da9-bcd4-5d33de4210e8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.207161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.207855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.208038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.208286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.208901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.211255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.225766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 921/1847] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:48.237950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 927/1848] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:47 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:48.242096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.245213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.246821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.258793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f660919-b271-4999-80f4-f2304bb45a50 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.265077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.265689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.266580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.266580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.267403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.269741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.286878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 928/1849] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:48.289308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.307576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067178b-8d57-42ad-9004-df0843228077 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.316506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.317858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.318209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.320063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.320978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.323546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.338692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 929/1850] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:48.339359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168f5d5-ea89-49a7-99ce-2221332ec754 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.343139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.350959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.351616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.351894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.352165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.359431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf594f-6e2d-4f9a-a29c-8bbb2df59123 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.364841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.365513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.365949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.366067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.366289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] PATH_INFO: `/v3/projects/a5ad555b260649fa8211fab1ba1860f7/users/010d07feb7c441f5bbbb53aa4e33cc47/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.367984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907842-0e07-44ba-97c9-ea887b3a1405 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.380855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 922/1851] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:48.401122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ad555b260649fa8211fab1ba1860f7, user_id=010d07feb7c441f5bbbb53aa4e33cc47, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.404673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.431023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 930/1852] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:48 2022] PUT /identity/v3/projects/a5ad555b260649fa8211fab1ba1860f7/users/010d07feb7c441f5bbbb53aa4e33cc47/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:48.443438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.444572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.445182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.514923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.517196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.567177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] MFA Rules not processed for user `010d07feb7c441f5bbbb53aa4e33cc47`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:48.567713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.667791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.679075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27bbcb-8b43-4226-b37a-e14401ff48de admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.689096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.690633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.691170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.691643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.693852 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:48.696807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:48.704162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 923/1853] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:48.709640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.717184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.719662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.738076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 931/1854] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:48.797970 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953076a-9ee3-404d-8df8-1811102a513c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.803090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.804306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.804747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.805038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.805776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.807703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.854415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 924/1855] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:48 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:48.856428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.867521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae7970-b476-4812-a1b9-26a55c3f3292 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.934088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.934730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.935452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.935452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.936022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.938093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:48.965812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 932/1856] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:48 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:48.970778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:48.982903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3410c6b-7250-4ead-a78f-30ce8290f3b8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:48.988908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:48.989753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:48.990154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:48.990521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:48.991308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:48.993027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.004951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 925/1857] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:49.009435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.019534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.020363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370d587-0c8f-454d-905f-a70b9c30a8f2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.021266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.025296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.025981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.026256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.026524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] PATH_INFO: `/v3/projects/115e0c8882554ef4a32f71678b652bc8/users/756af54a04144005b1b072d4328b3403/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.036088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c986-01c8-4e40-9f4a-578a556bd772 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.041212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.042051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.042406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.042754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] PATH_INFO: `/v3/projects/f2ffe7f4d7e2494590b50933e4612493` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.044012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ffe7f4d7e2494590b50933e4612493)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.046051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.055597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115e0c8882554ef4a32f71678b652bc8, user_id=756af54a04144005b1b072d4328b3403, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.062698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.079839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 933/1858] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:49 2022] PUT /identity/v3/projects/115e0c8882554ef4a32f71678b652bc8/users/756af54a04144005b1b072d4328b3403/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:49.083628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.101530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ffe7f4d7e2494590b50933e4612493', 'request_id': 'req-55130321-6a5c-481a-ac65-9c867a8b54f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.111313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ffe7f4d7e2494590b50933e4612493', 'request_id': 'req-55130321-6a5c-481a-ac65-9c867a8b54f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.120678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ffe7f4d7e2494590b50933e4612493', 'request_id': 'req-55130321-6a5c-481a-ac65-9c867a8b54f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.136031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] The token cache is being invalidate because project f2ffe7f4d7e2494590b50933e4612493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:49.136031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.137319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.142471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 926/1859] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:49 2022] DELETE /identity/v3/projects/f2ffe7f4d7e2494590b50933e4612493 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:49.143663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.145446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.158514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb9d06-5c88-4120-88b4-40b7b8536c19 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.222102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.224794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.225491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.225843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.226221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.226926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.227149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55130321-6a5c-481a-ac65-9c867a8b54f1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.228840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.235227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.236376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.236980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.237873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] PATH_INFO: `/v3/users/15b549b6e5bb4eb9836ab60cff94e215` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.238025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:49.239584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b549b6e5bb4eb9836ab60cff94e215)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.242459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.253914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 934/1860] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:49.255622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.257400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.273634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975a73-7aca-4ad8-b997-f7f47bc0aeac admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.279156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.279883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.280192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.280533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] PATH_INFO: `/v3/projects/115e0c8882554ef4a32f71678b652bc8/users/756af54a04144005b1b072d4328b3403/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.282663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115e0c8882554ef4a32f71678b652bc8, user_id=756af54a04144005b1b072d4328b3403, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.285540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fbd24-ea5b-4284-8028-b9cf68e80113 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.298924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 935/1861] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:49 2022] PUT /identity/v3/projects/115e0c8882554ef4a32f71678b652bc8/users/756af54a04144005b1b072d4328b3403/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:49.300552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b549b6e5bb4eb9836ab60cff94e215'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.309853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b549b6e5bb4eb9836ab60cff94e215'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.310052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.310181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.310438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.310946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b549b6e5bb4eb9836ab60cff94e215'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.343883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b549b6e5bb4eb9836ab60cff94e215'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.344194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] MFA Rules not processed for user `756af54a04144005b1b072d4328b3403`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:49.345428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.351507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b549b6e5bb4eb9836ab60cff94e215'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:49.358456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 927/1862] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:49 2022] DELETE /identity/v3/users/15b549b6e5bb4eb9836ab60cff94e215 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:49.421256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 936/1863] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:49.433162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.434642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.497973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89811fda-6ada-4ea2-b41c-06568998ef56 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:49.501607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:49.502225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:49.502500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:49.502767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:49.579123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:49.580576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88559711-4d91-4d8a-858f-2bea18767fc5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:49.591757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 928/1864] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:49.851049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:49.853118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:49.939271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d2e3f-c93e-49da-a4c2-dc79b08fa385 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.013357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.014315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.014487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.014828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] PATH_INFO: `/v3/projects/c761dc85044a485082565ced8b09fb16` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.016273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c761dc85044a485082565ced8b09fb16)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.018348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.045994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.120738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c761dc85044a485082565ced8b09fb16', 'request_id': 'req-19498095-267b-4ebe-82b5-403de8791b3c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:50.128596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c761dc85044a485082565ced8b09fb16', 'request_id': 'req-19498095-267b-4ebe-82b5-403de8791b3c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:50.148907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c761dc85044a485082565ced8b09fb16', 'request_id': 'req-19498095-267b-4ebe-82b5-403de8791b3c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:50.172848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] The token cache is being invalidate because project c761dc85044a485082565ced8b09fb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:50.173242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:50.174257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-19498095-267b-4ebe-82b5-403de8791b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:50.176072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 937/1865] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:49 2022] DELETE /identity/v3/projects/c761dc85044a485082565ced8b09fb16 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:50.180884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.181421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.181772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.206125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.208387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.232831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:50.296945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 929/1866] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:50.305894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.306463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.306463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.339611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.340880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.354085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 938/1867] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:50.360052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.372001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e5e95-86db-4d07-b502-c196f3739187 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.377719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.378432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.378781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.379974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.380678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:50.381174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.383689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.401352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 930/1868] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:50.405724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.419463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060926d-ebd7-4903-bd22-cffd0a5bd8c5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.427701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.428323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.428602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.428876 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.429511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.431845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.473075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 939/1869] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:50.477390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.488349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2881451-e5ce-4220-bb40-a8001ebb85ce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.557494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.558451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.558879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.559314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.560225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.562366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.593644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 931/1870] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:50.597719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.609564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84953763-e1f4-4e53-801c-6314161a05f7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.614998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.615607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.615887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.616541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.617306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.619172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.645892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 940/1871] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:50.656088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.669649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2aaeb1-683d-43b2-84f2-1e324e155add admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.675258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.676396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.676855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.679620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.681462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.683562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cbd09-af88-44bb-abd3-21532bda7bf8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.703014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 932/1872] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:50.709034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.723469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fb704-d073-4c45-951e-b4efda96bc3c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:50.737690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:50.738943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.742630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.742630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] PATH_INFO: `/v3/projects/96191133e1134e8aaae90b3cbdf8a0cc/users/a9958f7a9e9e4309ad622e77f6a2196c/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.760088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96191133e1134e8aaae90b3cbdf8a0cc, user_id=a9958f7a9e9e4309ad622e77f6a2196c, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:50.763557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:50.780202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 941/1873] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:50 2022] PUT /identity/v3/projects/96191133e1134e8aaae90b3cbdf8a0cc/users/a9958f7a9e9e4309ad622e77f6a2196c/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:50.785270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:50.785270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:50.785481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:50.807157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:50.809120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:50.920756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 933/1874] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:50.930561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:50.932663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.032875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb6824-58d3-4abf-907c-e35db4d3bef7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.040238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.040238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.040238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.040238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.056174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.059544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda2d63-c01c-4dc6-a2cb-47afa298342c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.076792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 942/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:51.254142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.255617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.269688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c098af1-3495-4610-b70b-c29b13edec34 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.274933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.275690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.276166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.276665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] PATH_INFO: `/v3/users/e72c69e3e9874295b2586e3f2c7c23ac` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.278268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72c69e3e9874295b2586e3f2c7c23ac)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.280719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.348641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.385096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72c69e3e9874295b2586e3f2c7c23ac'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.398264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72c69e3e9874295b2586e3f2c7c23ac'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.398994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72c69e3e9874295b2586e3f2c7c23ac'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.421009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72c69e3e9874295b2586e3f2c7c23ac'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.436062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72c69e3e9874295b2586e3f2c7c23ac'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.452943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 934/1876] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:51 2022] DELETE /identity/v3/users/e72c69e3e9874295b2586e3f2c7c23ac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:51.454516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.455147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.455147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.481732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:51.483219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.523090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.524606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.546292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.572850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 943/1877] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:51.587998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.589426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.616248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec91ad-cde4-45b1-b977-47d70acd5fec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.621862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.622264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.622804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.623106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] PATH_INFO: `/v3/users/010d07feb7c441f5bbbb53aa4e33cc47` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.625482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010d07feb7c441f5bbbb53aa4e33cc47)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.627607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.665008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580b5f5-67f2-44de-8e95-841f0a12644e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.671023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.671727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.672079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.672353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.688994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:51.690892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461d423-50b1-4699-bd38-596b7fb5efd6 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:51.700111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010d07feb7c441f5bbbb53aa4e33cc47'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.711309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 944/1878] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:51.713126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010d07feb7c441f5bbbb53aa4e33cc47'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.713847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010d07feb7c441f5bbbb53aa4e33cc47'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.730114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010d07feb7c441f5bbbb53aa4e33cc47'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.744781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010d07feb7c441f5bbbb53aa4e33cc47'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:51.756749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 935/1879] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:51 2022] DELETE /identity/v3/users/010d07feb7c441f5bbbb53aa4e33cc47 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:51.762805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9f50-1ce1-4325-b7e0-6a92c6cc2842 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.763470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9f50-1ce1-4325-b7e0-6a92c6cc2842 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.763925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9f50-1ce1-4325-b7e0-6a92c6cc2842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:51.784780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a7fb9f50-1ce1-4325-b7e0-6a92c6cc2842 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:51.786205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a7fb9f50-1ce1-4325-b7e0-6a92c6cc2842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.863057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 945/1880] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:56:51.884336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:51.891265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:51.931952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:51.981476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f506a-74cf-4c74-889a-488c943e1316 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:51.987704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:51.988950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:51.989378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:51.989795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.010279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:52.012838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:52.049280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 936/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:52.882518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:52.883133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:52.883133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:52.916445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] MFA Rules not processed for user `a9958f7a9e9e4309ad622e77f6a2196c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:52.917667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.015858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.017665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.042886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d53c3-28b3-4f4c-b161-2e993a22dff1 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.057094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.057757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.058036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.058317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] PATH_INFO: `/v3/projects/f0b514c56ad146499d3bbd78117e76ef` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.060719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b514c56ad146499d3bbd78117e76ef)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.080237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 946/1882] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:53.081111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.089122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.089122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.116316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d679c-78e4-4cf8-8516-1973a8c4637b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.126784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.127758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.128191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.128607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.129646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.133170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.181847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.210711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 947/1883] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:53 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:53.214015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.220090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.330651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b514c56ad146499d3bbd78117e76ef', 'request_id': 'req-0984cb16-18c7-4d61-bcb1-675c63ca5c13'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.342964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b514c56ad146499d3bbd78117e76ef', 'request_id': 'req-0984cb16-18c7-4d61-bcb1-675c63ca5c13'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.373348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e2cb-126b-4480-a458-42c174d45e1c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.373599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b514c56ad146499d3bbd78117e76ef', 'request_id': 'req-0984cb16-18c7-4d61-bcb1-675c63ca5c13'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.399279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] The token cache is being invalidate because project f0b514c56ad146499d3bbd78117e76ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:53.399864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.400733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.404326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 937/1884] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:53 2022] DELETE /identity/v3/projects/f0b514c56ad146499d3bbd78117e76ef => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:53.407523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.409293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.474070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.474934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.475314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.475581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] PATH_INFO: `/v3/projects/a5ad555b260649fa8211fab1ba1860f7` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.477164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ad555b260649fa8211fab1ba1860f7)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.479248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.522170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984cb16-18c7-4d61-bcb1-675c63ca5c13 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.530493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.531189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.531639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.531922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.532618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.534910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.565436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ad555b260649fa8211fab1ba1860f7', 'request_id': 'req-f94b93cc-673d-46db-87a3-69bd0817d662'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.570867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 938/1885] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:53 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:53.571713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.573477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.576246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ad555b260649fa8211fab1ba1860f7', 'request_id': 'req-f94b93cc-673d-46db-87a3-69bd0817d662'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.582885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ad555b260649fa8211fab1ba1860f7', 'request_id': 'req-f94b93cc-673d-46db-87a3-69bd0817d662'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.587307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437ca93-9452-4e48-9308-d8fd8e74a9d7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.601382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] The token cache is being invalidate because project a5ad555b260649fa8211fab1ba1860f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:53.601944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.602573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.604621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 948/1886] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:53 2022] DELETE /identity/v3/projects/a5ad555b260649fa8211fab1ba1860f7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:53.605189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.606840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.639212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.661885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.662586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.662712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.662801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.663289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] PATH_INFO: `/v3/users/8ca76cc1cf824bc4931fab6e2953c808` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.664297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca76cc1cf824bc4931fab6e2953c808)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.666501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.688213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b93cc-673d-46db-87a3-69bd0817d662 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.693088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.693686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.693981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.694233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.694869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.696548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.710759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 949/1887] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:53.710759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.710759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.714410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca76cc1cf824bc4931fab6e2953c808'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.723545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca76cc1cf824bc4931fab6e2953c808'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.724296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca76cc1cf824bc4931fab6e2953c808'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.724440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109217d-7dae-4e01-9758-34f65167874b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.729967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.730804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.731117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.731390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] PATH_INFO: `/v3/users/756af54a04144005b1b072d4328b3403` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.732587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756af54a04144005b1b072d4328b3403)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.734619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.737308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca76cc1cf824bc4931fab6e2953c808'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.744023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca76cc1cf824bc4931fab6e2953c808'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.753666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 939/1888] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:53 2022] DELETE /identity/v3/users/8ca76cc1cf824bc4931fab6e2953c808 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:53.753666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.755234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:53.794600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756af54a04144005b1b072d4328b3403'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.804514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756af54a04144005b1b072d4328b3403'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.805429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756af54a04144005b1b072d4328b3403'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.816780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756af54a04144005b1b072d4328b3403'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.833403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756af54a04144005b1b072d4328b3403'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:53.840847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 950/1889] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:53 2022] DELETE /identity/v3/users/756af54a04144005b1b072d4328b3403 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:53.857033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a39e1e-d292-46a5-ad8a-5301e420d982 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:53.929475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:53.930213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:53.930851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:53.930851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] PATH_INFO: `/v3/projects/b8917c8987fa47709a874fb7aef00133/users/a3d27e77e0734d33b5fcfd58676b67bd/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:53.945539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:53.953379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8917c8987fa47709a874fb7aef00133, user_id=a3d27e77e0734d33b5fcfd58676b67bd, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:53.957602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:53.979503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 940/1890] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:53 2022] PUT /identity/v3/projects/b8917c8987fa47709a874fb7aef00133/users/a3d27e77e0734d33b5fcfd58676b67bd/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:53.984138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:53.985822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.005313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855895ff-461b-4983-a2bc-6653262f1591 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.137577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.138393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.138815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.142826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.143907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.146140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.163418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 951/1891] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:54.167622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.185695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4028b-f079-44e4-9ae7-0b27bf3bda7f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.191885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.192569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.192981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.193276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] PATH_INFO: `/v3/projects/b8917c8987fa47709a874fb7aef00133/users/a3d27e77e0734d33b5fcfd58676b67bd/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.195615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8917c8987fa47709a874fb7aef00133, user_id=a3d27e77e0734d33b5fcfd58676b67bd, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.198283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.216272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 941/1892] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:54 2022] PUT /identity/v3/projects/b8917c8987fa47709a874fb7aef00133/users/a3d27e77e0734d33b5fcfd58676b67bd/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:54.273185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.274239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.289695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db5094-2c61-45f6-82e7-e6f7b20b98b8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.346958 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.348036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.349821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.422216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.423447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.424680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.425136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] PATH_INFO: `/v3/projects/11974f4e6a9d4d0992815fdbf2788633` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.428235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=11974f4e6a9d4d0992815fdbf2788633)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.431433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.525297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedf8e8-b23c-4e78-8e21-ca9da452f217 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.528706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11974f4e6a9d4d0992815fdbf2788633', 'request_id': 'req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.544282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11974f4e6a9d4d0992815fdbf2788633', 'request_id': 'req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.556535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11974f4e6a9d4d0992815fdbf2788633', 'request_id': 'req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.583230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] The token cache is being invalidate because project 11974f4e6a9d4d0992815fdbf2788633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:54.583593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.585427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.588195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 952/1893] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:54 2022] DELETE /identity/v3/projects/11974f4e6a9d4d0992815fdbf2788633 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:54.593074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.596200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.661187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.661828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.662115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.662385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] PATH_INFO: `/v3/projects/115e0c8882554ef4a32f71678b652bc8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.665957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115e0c8882554ef4a32f71678b652bc8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.668104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.743723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbf2e-ad4d-4634-98ea-a1c34e931adc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.751240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115e0c8882554ef4a32f71678b652bc8', 'request_id': 'req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.768777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115e0c8882554ef4a32f71678b652bc8', 'request_id': 'req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.782275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115e0c8882554ef4a32f71678b652bc8', 'request_id': 'req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.799331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] The token cache is being invalidate because project 115e0c8882554ef4a32f71678b652bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:54.799646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.800368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e38e3eb-cf8d-423e-b30a-3fef0d773dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.802390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 942/1894] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:54 2022] DELETE /identity/v3/projects/115e0c8882554ef4a32f71678b652bc8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:54.807965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109bc9-1dff-4f32-b7c0-9d34877f745a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.808403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109bc9-1dff-4f32-b7c0-9d34877f745a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.808786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b109bc9-1dff-4f32-b7c0-9d34877f745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.835264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.835625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4b109bc9-1dff-4f32-b7c0-9d34877f745a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:54.835972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.836263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.836546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] PATH_INFO: `/v3/users/441b9ee75dba4a2f87475d6687a27940` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.837749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=441b9ee75dba4a2f87475d6687a27940)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.837992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4b109bc9-1dff-4f32-b7c0-9d34877f745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.840393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.857594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 943/1895] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:54.859577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1215-5829-49fe-afc7-94c0e1be09cd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.860005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1215-5829-49fe-afc7-94c0e1be09cd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.860463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce1215-5829-49fe-afc7-94c0e1be09cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.866429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.885927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:54.893307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2cce1215-5829-49fe-afc7-94c0e1be09cd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:54.894870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2cce1215-5829-49fe-afc7-94c0e1be09cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:54.911513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 944/1896] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:54.922118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68073a-7a7f-4651-96e2-b6070dedf018 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.922902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68073a-7a7f-4651-96e2-b6070dedf018 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.925522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68073a-7a7f-4651-96e2-b6070dedf018 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.929704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441b9ee75dba4a2f87475d6687a27940'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.930399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 945/1897] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:56:54 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:56:54.930848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68073a-7a7f-4651-96e2-b6070dedf018 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:54.945693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441b9ee75dba4a2f87475d6687a27940'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.947234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441b9ee75dba4a2f87475d6687a27940'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.957944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68073a-7a7f-4651-96e2-b6070dedf018 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:54.970775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441b9ee75dba4a2f87475d6687a27940'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:54.977582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:54.978202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:54.978481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:54.978752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:54.980811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:54.981572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:54.985893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:54.992957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-679e8dfa-5585-4eca-8f35-8b75af83ac28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441b9ee75dba4a2f87475d6687a27940'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.003375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 953/1898] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:56:54 2022] DELETE /identity/v3/users/441b9ee75dba4a2f87475d6687a27940 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:55.006141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.006540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.006688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.014164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 946/1899] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:55.027272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.035585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.077748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:55.077953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc2e93-e74a-4e7b-9b5f-c4a64ef2a33e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.078905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.211129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.211831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.212133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.212317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.212941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.224013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.239023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 954/1900] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:55.261702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.264047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.285833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 947/1901] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:55.286734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.308743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb1596-12dd-4d89-94a4-13749747b165 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.439490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8898a04-4b63-43c5-9d3e-a09404b81f00 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.448349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.449895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.450435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.450937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.451516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.452493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.454200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.456979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.457448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.460838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.471790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.473843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.493110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 955/1902] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:55.494631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.497014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.522403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456dbaa-c8f2-4edf-a183-7a5177336642 service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.528056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 948/1903] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:55.529328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.530054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.530328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.530676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] PATH_INFO: `/v3/projects/bbabc9acc01f44fbbcf484be75f74a48` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.532042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.532226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbabc9acc01f44fbbcf484be75f74a48)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.534235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.545449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:55.553612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46cde8-7c6c-4943-a078-842ffd333902 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.573110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.573983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.574372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.574836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.578503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.581536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.603468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbabc9acc01f44fbbcf484be75f74a48', 'request_id': 'req-781f8347-94fa-4fda-9c81-eddf20e43932'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.608215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 949/1904] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:56:55.612280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.616631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbabc9acc01f44fbbcf484be75f74a48', 'request_id': 'req-781f8347-94fa-4fda-9c81-eddf20e43932'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.624719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbabc9acc01f44fbbcf484be75f74a48', 'request_id': 'req-781f8347-94fa-4fda-9c81-eddf20e43932'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.627907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dcf15-2f0e-429f-8597-4cdbb64afbe9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.647583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] The token cache is being invalidate because project bbabc9acc01f44fbbcf484be75f74a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:56:55.647913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.648735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-781f8347-94fa-4fda-9c81-eddf20e43932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:56:55.650954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 956/1905] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:56:55 2022] DELETE /identity/v3/projects/bbabc9acc01f44fbbcf484be75f74a48 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:55.656364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78ad70-a4c4-48b2-9b17-d6189e0d8b18 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.656647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78ad70-a4c4-48b2-9b17-d6189e0d8b18 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.656914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78ad70-a4c4-48b2-9b17-d6189e0d8b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.682039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6b78ad70-a4c4-48b2-9b17-d6189e0d8b18 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:55.683602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6b78ad70-a4c4-48b2-9b17-d6189e0d8b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.686317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:55.703113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.705226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.705704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.706283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.707322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.709373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.721264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 950/1906] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:55.725749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.729422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.753137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269c37-c082-4d1a-8ae4-2e3180cc5fa7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.766046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.767985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.768301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.768624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] PATH_INFO: `/v3/projects/2160b8a8dd4f4926804243f2aaefdd23/users/d76d08720ed54c9ea97e746d475f4b8d/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.784903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 957/1907] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:55.787310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.787575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.787995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.789287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2160b8a8dd4f4926804243f2aaefdd23, user_id=d76d08720ed54c9ea97e746d475f4b8d, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.792518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b7d87-523d-4b5b-a575-fc2223bf2154 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:55.813284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 951/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:55 2022] PUT /identity/v3/projects/2160b8a8dd4f4926804243f2aaefdd23/users/d76d08720ed54c9ea97e746d475f4b8d/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:55.815865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:55.817109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.823742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.825636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.826155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.872942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] MFA Rules not processed for user `d76d08720ed54c9ea97e746d475f4b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:55.875503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:55.902065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 958/1909] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:55.906802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:55.944313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eff3f-f00a-490b-8ff3-0cd8e1629d88 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:55.957891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:55.959445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:55.959750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:55.979102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:55.979936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:56:55.982957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:55.990090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.008259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 952/1910] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:56.016136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.022644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 959/1911] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:56:56.024205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.035792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298303bd-47f6-45e5-95fa-4a905a28094e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.046002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.046852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.047187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.047463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.048295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.048823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1780e0-89eb-49b5-8922-61568b5a0197 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.052113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66df35-c5fa-4e0e-81dc-7fcbe8c97d0e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.076184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.077063 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.077546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.078031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.078864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.081621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.115467 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 953/1912] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:56.135631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.138151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.138459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.151716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 960/1913] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:56.155598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.165562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:56.196312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] MFA Rules not processed for user `a3d27e77e0734d33b5fcfd58676b67bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:56.197552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:56.327124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 954/1914] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:56.328792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.329315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd758898-2e45-43ae-8697-7600c5e2749d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.344341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.344962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.345245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.345514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.346137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.348090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.352002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc69e82-f0e6-4eaf-953b-434ef8ebe483 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.358824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.359935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.360423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.360900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.361733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.363980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.391329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 961/1915] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:56.392297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.395449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:56.396653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:56.401703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 955/1916] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:56 2022] POST /identity/v3/users => generated 292 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:56.404356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.432610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba11d9f-2446-4cc1-abb4-43ef6527b306 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.439194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.441139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.441280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.441492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.442148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.445110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.479593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 956/1917] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:56.482344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.495392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692cfa3-42a5-4e1f-a259-23a96fb37c8e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.505545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.506305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.506776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.507286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.516411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.518878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.535281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 957/1918] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:56.537767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fddd76-5a4a-4c31-9852-6d92ac49751f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.539367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.549120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.550329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.550680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.551216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.556641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:56.568105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb638-d59b-4099-b049-e6bc22d7d77b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.583886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.584564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.586380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.586669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] PATH_INFO: `/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa/users/85df55db57bb486f8cc01a8a9cd738e2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.630936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359f7a136f5046b99fe9ad4a7498a1aa, user_id=85df55db57bb486f8cc01a8a9cd738e2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.635228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.675262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 958/1919] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:56 2022] PUT /identity/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa/users/85df55db57bb486f8cc01a8a9cd738e2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:56.677095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.695708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.697438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.711431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 962/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:56:56.712658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.728541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037f8d7-7aae-4863-986a-48f06bebd7eb service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.838590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:56.857313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886583b5-22e1-4df7-b663-7e7374468017 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.861569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.862487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.862863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.863527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.864518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.867617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.869881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.871034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.871328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.871633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.872268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.876946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.893549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 963/1921] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:56.894663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.911753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 959/1922] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:56.916214 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:56.926730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0439a-b2a6-4905-9ce0-c3be8d26c1d1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.940584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.944565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.944850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.945118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] PATH_INFO: `/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa/users/85df55db57bb486f8cc01a8a9cd738e2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.946274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523aefa-6b93-48cf-8488-f8676376555b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:56.952349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359f7a136f5046b99fe9ad4a7498a1aa, user_id=85df55db57bb486f8cc01a8a9cd738e2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:56.957738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3848e98-e30a-4885-af0a-87fb11182ce6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:56.962228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:56.963552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:56.963839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:56.964432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] PATH_INFO: `/v3/projects/42b6514d1b8d4aaf9077928da5784338/users/71a2f640832249f18c2d4afd960bd0ab/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:56.990883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 964/1923] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:56 2022] PUT /identity/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa/users/85df55db57bb486f8cc01a8a9cd738e2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:57.002860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.003250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.004187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.005122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b6514d1b8d4aaf9077928da5784338, user_id=71a2f640832249f18c2d4afd960bd0ab, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.033258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82320a35-8182-4cc0-bcdc-0b834f88b482 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.075830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 960/1924] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:56 2022] PUT /identity/v3/projects/42b6514d1b8d4aaf9077928da5784338/users/71a2f640832249f18c2d4afd960bd0ab/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:57.087634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.087919 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.088192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.119101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] MFA Rules not processed for user `85df55db57bb486f8cc01a8a9cd738e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:57.126169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.129898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] MFA Rules not processed for user `71a2f640832249f18c2d4afd960bd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:57.131160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.300277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 961/1925] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:57.301222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.322929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 965/1926] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:57.335502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.337308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.342519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.426502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57147-971e-4929-843f-2dd5bbe5c155 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.431691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.432319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.432594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.432953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.448859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a97ac-3375-4fce-87e9-6d8e67d5d53a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.458454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.459307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.459653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.459988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.460760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.467296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.521529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 962/1927] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:57.521724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.522749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.523127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.533862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 966/1928] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:57.592892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca408fbf-e5c3-4453-b8d3-3971a1fb92f2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.597742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.598388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.598670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.598941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.599628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.601549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.630434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 963/1929] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:57.632157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.644424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbba0aa-8333-4bca-a20a-988a61251ae9 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.650457 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.651288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.651736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.652205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.653103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.654858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.657573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:57.670648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 967/1930] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:57.675025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.686158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2b2a-9c58-4d1d-9aed-80ec2eb8e2b1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.692618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.693508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.693806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.694116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.694713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.696351 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e38773-8126-47b8-bbdb-9152bb6a42e0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.709825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 964/1931] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:57.714245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:57.727215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afcb6b-0f17-4ccd-910b-9da5bdbf01b2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:57.733412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:57.734105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.734398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.734725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] PATH_INFO: `/v3/projects/ee8a2307eb3b4d8fb1baacb231c4a008/users/023e846910fd4b648d59bec5932ce910/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.755511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8a2307eb3b4d8fb1baacb231c4a008, user_id=023e846910fd4b648d59bec5932ce910, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:57.758105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:57.777476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 968/1932] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:57 2022] PUT /identity/v3/projects/ee8a2307eb3b4d8fb1baacb231c4a008/users/023e846910fd4b648d59bec5932ce910/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:56:57.790845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:57.791225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:57.794077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:57.832291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] MFA Rules not processed for user `023e846910fd4b648d59bec5932ce910`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:57.833525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:57.952588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 965/1933] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:57.957438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.041610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b591-250f-4878-9597-c559ed45cac5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.046335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.046940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.047285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.047285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.048155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.050062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.101411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 969/1934] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:56:57 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:56:58.102591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.114456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb4bc8-7792-4c9d-af9b-878dd68d7231 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.118367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.178102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.178660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.178936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.179234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.179869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.182361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.208421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 966/1935] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:56:58 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:56:58.209585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.222076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8feb71-5139-4df7-a1a3-7e177d5f61a3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.226688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.227323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.227660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.227933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.228619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.230112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.241873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 970/1936] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:58.246354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.258725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780aec2e-b466-475a-9be6-0eacf4df7e3b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.263707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.264374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.264663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.264942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] PATH_INFO: `/v3/projects/8d8e7ca3597e40598b9930eb0660b449/users/577dc96683a6460297966ee5f41b27e5/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.287501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8e7ca3597e40598b9930eb0660b449, user_id=577dc96683a6460297966ee5f41b27e5, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.290211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.313067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 967/1937] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:58 2022] PUT /identity/v3/projects/8d8e7ca3597e40598b9930eb0660b449/users/577dc96683a6460297966ee5f41b27e5/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:58.315007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.327793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3f3d0-2522-488e-9a2a-24718165ae6e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.400127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.400731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.401480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.401480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.402578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.404949 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b81771-3716-4a77-9b56-183fbd93ad1d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.415565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 971/1938] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:56:58.420266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.431024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907318d-d935-47e8-b95b-8e112e0171a7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.435654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.436396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.436738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.437012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] PATH_INFO: `/v3/projects/8d8e7ca3597e40598b9930eb0660b449/users/577dc96683a6460297966ee5f41b27e5/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.439036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d8e7ca3597e40598b9930eb0660b449, user_id=577dc96683a6460297966ee5f41b27e5, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.441938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.465472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 968/1939] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:56:58 2022] PUT /identity/v3/projects/8d8e7ca3597e40598b9930eb0660b449/users/577dc96683a6460297966ee5f41b27e5/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:56:58.467279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.467574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.467855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.502363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] MFA Rules not processed for user `577dc96683a6460297966ee5f41b27e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:56:58.503606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:56:58.615186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:56:58.647983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.691827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 972/1940] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:56:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:56:58.704805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:56:58.775710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9070058-88d9-484b-9f51-dfd15ca5c613 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.780204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.780826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.781102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.781373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.816099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4efab-b636-4c9c-ac69-5ff9baf20f6d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:56:58.820931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:56:58.824968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:56:58.825290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:56:58.825577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:56:58.874664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:58.876481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:58.888112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 969/1941] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:56:59.003664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:56:59.005300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:56:59.021010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 973/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:56:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:03.701828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:03.710334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303122b-d671-45ac-96ed-64fd55913448 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:03.714176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:03.714818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:03.715254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:03.715535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:03.721839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:03.812303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:03.812303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:03.823705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 970/1943] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:04.190534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.206064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1f642-0adc-466f-8023-413ce0df1e58 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.210307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.211341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.211798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.212041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.292002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.294525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.305882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 974/1944] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:04.950143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:04.958766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cae0fd-9c26-4c3a-93c7-99089fe3f2e9 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:04.962392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:04.963001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:04.963293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:04.963567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:04.974008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:04.975494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4856d73-76a8-439f-85f6-bf1274a024de service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:04.988671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 971/1945] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:14.300817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:14.985564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:15.259492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:15.325700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff14371-f866-427c-afa0-16ca0cc462ac service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:15.330313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:15.330950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.331292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.331566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] PATH_INFO: `/v3/users/71a2f640832249f18c2d4afd960bd0ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.332727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a2f640832249f18c2d4afd960bd0ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:15.334677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:15.394851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a2f640832249f18c2d4afd960bd0ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:15.403882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a2f640832249f18c2d4afd960bd0ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:15.405316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a2f640832249f18c2d4afd960bd0ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:15.414344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a2f640832249f18c2d4afd960bd0ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:15.443927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a2f640832249f18c2d4afd960bd0ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:15.452460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 975/1946] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:15 2022] DELETE /identity/v3/users/71a2f640832249f18c2d4afd960bd0ab => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:15.456909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.457179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.457464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.484690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:15.486002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:15.560306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 972/1947] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:15.570750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:15.572298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:15.660322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99b82d-dea2-4f12-b36c-e431cd94a9a9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:15.664289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:15.664862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:15.665143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:15.665419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:15.676817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:15.678862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:15.701642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 976/1948] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:16.192517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.194196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:16.216519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400f664-63e3-4670-9379-b95d6631fec7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.235627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.236352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.236755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.237038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] PATH_INFO: `/v3/projects/42b6514d1b8d4aaf9077928da5784338` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.238284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b6514d1b8d4aaf9077928da5784338)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.250756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.314436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b6514d1b8d4aaf9077928da5784338', 'request_id': 'req-18ac2740-b19c-4528-8261-a7432c931a88'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.324296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b6514d1b8d4aaf9077928da5784338', 'request_id': 'req-18ac2740-b19c-4528-8261-a7432c931a88'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.334493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b6514d1b8d4aaf9077928da5784338', 'request_id': 'req-18ac2740-b19c-4528-8261-a7432c931a88'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.348748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] The token cache is being invalidate because project 42b6514d1b8d4aaf9077928da5784338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:16.349199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.349878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.351747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 973/1949] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:16 2022] DELETE /identity/v3/projects/42b6514d1b8d4aaf9077928da5784338 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:16.356078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.368016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dc6d6-1615-4136-a79c-2c1123111baa service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:16.393525 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:16.444104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:16.445754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:16.445754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:16.445754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] PATH_INFO: `/v3/users/023e846910fd4b648d59bec5932ce910` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:16.448178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=023e846910fd4b648d59bec5932ce910)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:16.449841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:16.519612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023e846910fd4b648d59bec5932ce910'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.535144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023e846910fd4b648d59bec5932ce910'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.535909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023e846910fd4b648d59bec5932ce910'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.547959 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023e846910fd4b648d59bec5932ce910'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.561581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023e846910fd4b648d59bec5932ce910'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:16.567888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 977/1950] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:16 2022] DELETE /identity/v3/users/023e846910fd4b648d59bec5932ce910 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:16.990583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:16.992498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:17.027256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac2740-b19c-4528-8261-a7432c931a88 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.094633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.095306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.095581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.095855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] PATH_INFO: `/v3/projects/ee8a2307eb3b4d8fb1baacb231c4a008` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.096969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8a2307eb3b4d8fb1baacb231c4a008)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.098856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.131015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.152399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8a2307eb3b4d8fb1baacb231c4a008', 'request_id': 'req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.162716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8a2307eb3b4d8fb1baacb231c4a008', 'request_id': 'req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.170933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8a2307eb3b4d8fb1baacb231c4a008', 'request_id': 'req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.185072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] The token cache is being invalidate because project ee8a2307eb3b4d8fb1baacb231c4a008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:17.185594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.186676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.188485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 974/1951] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:16 2022] DELETE /identity/v3/projects/ee8a2307eb3b4d8fb1baacb231c4a008 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:17.192444 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.193892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:17.207475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d0d5a-67d8-4dc0-8a0f-336c502cbd7d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.287339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.287898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.288174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.288442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] PATH_INFO: `/v3/users/577dc96683a6460297966ee5f41b27e5` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.289558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=577dc96683a6460297966ee5f41b27e5)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.291496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:17.339305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:17.343206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577dc96683a6460297966ee5f41b27e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.352505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577dc96683a6460297966ee5f41b27e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.353194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577dc96683a6460297966ee5f41b27e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.361039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577dc96683a6460297966ee5f41b27e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.375091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-eab57264-a5c0-4147-9cee-7598d1f2d344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577dc96683a6460297966ee5f41b27e5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:17.381190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 978/1952] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:17 2022] DELETE /identity/v3/users/577dc96683a6460297966ee5f41b27e5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:17.821979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:17.823918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:17.838745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fbbc0-e0a9-4be3-84e7-23d12746ce37 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:17.939650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:17.940397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:17.940825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:17.941107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] PATH_INFO: `/v3/projects/8d8e7ca3597e40598b9930eb0660b449` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:17.942390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d8e7ca3597e40598b9930eb0660b449)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:17.944401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.006704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d8e7ca3597e40598b9930eb0660b449', 'request_id': 'req-eadb1d3f-b37a-4ac0-b384-67debc2baa45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:18.015991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d8e7ca3597e40598b9930eb0660b449', 'request_id': 'req-eadb1d3f-b37a-4ac0-b384-67debc2baa45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:18.026222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d8e7ca3597e40598b9930eb0660b449', 'request_id': 'req-eadb1d3f-b37a-4ac0-b384-67debc2baa45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:18.037988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] The token cache is being invalidate because project 8d8e7ca3597e40598b9930eb0660b449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:18.038377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:18.039164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eadb1d3f-b37a-4ac0-b384-67debc2baa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:18.040978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 975/1953] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:17 2022] DELETE /identity/v3/projects/8d8e7ca3597e40598b9930eb0660b449 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:18.047708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.048073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.048428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.073687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:18.075708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:18.136908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 979/1954] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:18.145250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.145541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.145813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.167414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:18.168655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:18.182614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 976/1955] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:18.187051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.196899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a642-8dc4-4c63-b928-6c5fe5a49c86 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.202643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.203410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.203748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.204074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.206024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:18.206024 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.207139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.225864 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 980/1956] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:18.227093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.236242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228090c-9c98-483a-a76d-d9d6a90293ca None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.241160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.241632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.241923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.242177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.242842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.244756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.275713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 977/1957] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:18 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:18.280649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.292119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b4f2-2187-4991-a684-3ad29288af73 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.368832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.369594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.369712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.369980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.370827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.373136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.395502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 981/1958] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:18 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:18.399742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.414195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1828e-7a81-4176-ad35-44ea9ea65dbe admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.418995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.419640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.419923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.420199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.420859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.422490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.433860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 978/1959] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:18.438553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.449015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a6156-9bd1-4c85-8b7f-4db1f71d648e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.454881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.455571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.455848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.456114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.456749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.458341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f093-30e0-427d-b494-e2e4864266c0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.473083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 982/1960] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:18.474484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.480980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.487135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9f1db-9046-45a7-92da-2eb44a5a39a7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.493457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.494350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.494618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.494885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] PATH_INFO: `/v3/projects/4cd933e2a6414ec99b51d46aa131d595/users/d39bab1f36d24e008120ba7f066f6241/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.510807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd933e2a6414ec99b51d46aa131d595, user_id=d39bab1f36d24e008120ba7f066f6241, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.514180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.527845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 979/1961] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:18 2022] PUT /identity/v3/projects/4cd933e2a6414ec99b51d46aa131d595/users/d39bab1f36d24e008120ba7f066f6241/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:18.537891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68a00-8bf9-4f14-877c-3ff3f5d5c7bd None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.538187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68a00-8bf9-4f14-877c-3ff3f5d5c7bd None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.538464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68a00-8bf9-4f14-877c-3ff3f5d5c7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.567292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-64f68a00-8bf9-4f14-877c-3ff3f5d5c7bd None None] MFA Rules not processed for user `d39bab1f36d24e008120ba7f066f6241`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:18.568527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-64f68a00-8bf9-4f14-877c-3ff3f5d5c7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:18.583289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:18.640180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 983/1962] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:18.650840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:18.652576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:18.711886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c958-afac-436e-9a34-ba83730decd8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:18.715497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:18.716120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:18.716380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:18.716646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:18.725666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:18.727557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b49e2-b733-4886-9663-72487ef91f40 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:18.737484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 980/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:28.636723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=20 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:28.732875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:40.191844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.192417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.192417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.193742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 984/1964] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:40.222308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4deb46-0fcb-48e4-bb6e-437c5b10880a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.223113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4deb46-0fcb-48e4-bb6e-437c5b10880a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.223406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4deb46-0fcb-48e4-bb6e-437c5b10880a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.243636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7c4deb46-0fcb-48e4-bb6e-437c5b10880a None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:40.245204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7c4deb46-0fcb-48e4-bb6e-437c5b10880a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:40.328571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 981/1965] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:57:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:40.342801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.344842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:40.361190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9b4a9-0d0d-4c67-9802-91fa526a6f75 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.366025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.366647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.366925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.367218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.376227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.377803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62aff2-458d-4571-bc3b-bf472512eaba service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.390243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 985/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:57:40.830796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.831121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.831408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.832895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 982/1967] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:57:40.870365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.870899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.871333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.893565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] MFA Rules not processed for user `5b69ade20aca4860a575feb79522e9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:40.894792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:40.904845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 986/1968] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:57:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:40.907047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.908840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:40.920626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:40.935503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee0bed-6057-461a-abd1-20ddf418e613 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.941658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.942264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.942535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.942803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.951835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.953427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:40.964516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 987/1969] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:57:40.986917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b661534-b09f-493e-bfea-87d763c65ee6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:40.992133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:40.992771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:40.993050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:40.993424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] PATH_INFO: `/v3/users/a9958f7a9e9e4309ad622e77f6a2196c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:40.994529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9958f7a9e9e4309ad622e77f6a2196c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:40.996443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.049244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9958f7a9e9e4309ad622e77f6a2196c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.057446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9958f7a9e9e4309ad622e77f6a2196c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.058440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9958f7a9e9e4309ad622e77f6a2196c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.072574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9958f7a9e9e4309ad622e77f6a2196c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.079789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9958f7a9e9e4309ad622e77f6a2196c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.085692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 983/1970] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:40 2022] DELETE /identity/v3/users/a9958f7a9e9e4309ad622e77f6a2196c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:41.265975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.276990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396239-7f19-4f97-97c9-2bdfe2972f52 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.344888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.345583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.345906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.346139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] PATH_INFO: `/v3/projects/96191133e1134e8aaae90b3cbdf8a0cc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.347484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96191133e1134e8aaae90b3cbdf8a0cc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.349429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.402017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96191133e1134e8aaae90b3cbdf8a0cc', 'request_id': 'req-79e1926f-72bf-443b-859c-dc15952dbb1a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.410315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96191133e1134e8aaae90b3cbdf8a0cc', 'request_id': 'req-79e1926f-72bf-443b-859c-dc15952dbb1a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.417561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96191133e1134e8aaae90b3cbdf8a0cc', 'request_id': 'req-79e1926f-72bf-443b-859c-dc15952dbb1a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.434028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] The token cache is being invalidate because project 96191133e1134e8aaae90b3cbdf8a0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:41.434318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.434947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.436474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 988/1971] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:41 2022] DELETE /identity/v3/projects/96191133e1134e8aaae90b3cbdf8a0cc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:41.441099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.442463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:41.456192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb046bb-9b2e-403e-bf60-548e7bdc28c0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.469637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.526740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.527702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.528518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.528794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] PATH_INFO: `/v3/users/a3d27e77e0734d33b5fcfd58676b67bd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.529944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d27e77e0734d33b5fcfd58676b67bd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.531947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.587332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d27e77e0734d33b5fcfd58676b67bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.599273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d27e77e0734d33b5fcfd58676b67bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.600360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d27e77e0734d33b5fcfd58676b67bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.614508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d27e77e0734d33b5fcfd58676b67bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.621487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d27e77e0734d33b5fcfd58676b67bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.627710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 984/1972] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:41 2022] DELETE /identity/v3/users/a3d27e77e0734d33b5fcfd58676b67bd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:41.747654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.749416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:41.764762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1926f-72bf-443b-859c-dc15952dbb1a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.828614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:41.831616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:41.841384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.853567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.854904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.855510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.856140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] PATH_INFO: `/v3/users/d76d08720ed54c9ea97e746d475f4b8d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.858475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76d08720ed54c9ea97e746d475f4b8d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.863299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.920525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f7b8-9574-4ae6-903a-803549836d1e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:41.925249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:41.925884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.926336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.926687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] PATH_INFO: `/v3/projects/b8917c8987fa47709a874fb7aef00133` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.928185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8917c8987fa47709a874fb7aef00133)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:41.931163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:41.933825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76d08720ed54c9ea97e746d475f4b8d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.942611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76d08720ed54c9ea97e746d475f4b8d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.943424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76d08720ed54c9ea97e746d475f4b8d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.952148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76d08720ed54c9ea97e746d475f4b8d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.967524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-22fcec9e-02c1-44e3-b217-ae35ddb78a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76d08720ed54c9ea97e746d475f4b8d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.974162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 989/1973] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:41 2022] DELETE /identity/v3/users/d76d08720ed54c9ea97e746d475f4b8d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:41.975485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:41.979648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:41.980293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:41.980705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:41.988902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8917c8987fa47709a874fb7aef00133', 'request_id': 'req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:41.998362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8917c8987fa47709a874fb7aef00133', 'request_id': 'req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.006322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:42.007577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.009950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8917c8987fa47709a874fb7aef00133', 'request_id': 'req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.021202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] The token cache is being invalidate because project b8917c8987fa47709a874fb7aef00133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:42.021500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.022193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-03a93612-d0b8-4def-b1eb-c7b9e0294a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.023743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 985/1974] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:41 2022] DELETE /identity/v3/projects/b8917c8987fa47709a874fb7aef00133 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:42.029108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ca476-9880-4d49-9b32-c997a1718da9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.029395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ca476-9880-4d49-9b32-c997a1718da9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.029669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ca476-9880-4d49-9b32-c997a1718da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.052076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9f6ca476-9880-4d49-9b32-c997a1718da9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:42.053309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9f6ca476-9880-4d49-9b32-c997a1718da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.074436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 990/1975] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:42.085704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.087104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.116461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 986/1976] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:42.125228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.125532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.125918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.145856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:42.147087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.153745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505efcc-7202-4781-bcbd-6fee44ceaa32 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.156682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 987/1977] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:42.157551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.158256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.158604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.158991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.159621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.168064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.168799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0973a4c-0774-426e-908e-9199d381fe46 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.169518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.173360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.173936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.174215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.174482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.175150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:42.175450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.177011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.178703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 991/1978] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:42.188769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 988/1979] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:42.192996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.203359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f663f-ada4-485c-b38b-51b1753d5e03 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.208032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.208700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.209052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.209418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.210123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.212275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.235540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.245970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 992/1980] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:42.250640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.261508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90fc1-64d6-4c08-933e-7004977be6ff admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.344451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.345284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.345600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.346247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.346469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.348453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.373496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 989/1981] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:42.374703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.385282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4cf5f-5d6b-4e57-b450-117bcc247661 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.391460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.392140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.392487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.392768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.393410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.395044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.406090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 993/1982] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:42.410154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.420224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917962dc-6153-4e26-9bbc-a55a3dacba5a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.424760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.425339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.425809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.425981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.426564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.428227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.439965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 990/1983] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:42.444296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.455802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb365f-6919-4618-8772-f783e93ff0a7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.460496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.461185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.461481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.462350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] PATH_INFO: `/v3/projects/4a5183095ea940b49083d47f986be096/users/1e38740579694e388b1f17b42b946bf7/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.468959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.471401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.480093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5183095ea940b49083d47f986be096, user_id=1e38740579694e388b1f17b42b946bf7, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.483138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4622076-ea6b-44f2-a0fd-ac20ab3fa742 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.488653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788487c7-dfe6-4895-9383-b1f5f429e5dd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.493780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.494567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.494881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.495046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] PATH_INFO: `/v3/projects/2160b8a8dd4f4926804243f2aaefdd23` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.496193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2160b8a8dd4f4926804243f2aaefdd23)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.499018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.499236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 994/1984] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:42 2022] PUT /identity/v3/projects/4a5183095ea940b49083d47f986be096/users/1e38740579694e388b1f17b42b946bf7/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:42.509973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.510260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.510534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.543581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] MFA Rules not processed for user `1e38740579694e388b1f17b42b946bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:42.543775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.544914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.577640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2160b8a8dd4f4926804243f2aaefdd23', 'request_id': 'req-b04f430e-76b1-4e4c-91aa-30ddf29d821e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.586197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2160b8a8dd4f4926804243f2aaefdd23', 'request_id': 'req-b04f430e-76b1-4e4c-91aa-30ddf29d821e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.595914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2160b8a8dd4f4926804243f2aaefdd23', 'request_id': 'req-b04f430e-76b1-4e4c-91aa-30ddf29d821e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.610642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] The token cache is being invalidate because project 2160b8a8dd4f4926804243f2aaefdd23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:42.611238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.612006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.613969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 991/1985] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:42 2022] DELETE /identity/v3/projects/2160b8a8dd4f4926804243f2aaefdd23 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:42.617990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.621948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.647005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 995/1986] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:42.659512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.661400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.718123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f430e-76b1-4e4c-91aa-30ddf29d821e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.723351 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.723921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.724301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.724679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] PATH_INFO: `/v3/users/85df55db57bb486f8cc01a8a9cd738e2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.726157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85df55db57bb486f8cc01a8a9cd738e2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.728881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.743953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b1e63-d0f8-44ff-ae47-63b547b1e0db None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:42.805623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85df55db57bb486f8cc01a8a9cd738e2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.807930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:42.808642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:42.809611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:42.810013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:42.812225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.815646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85df55db57bb486f8cc01a8a9cd738e2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.816321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85df55db57bb486f8cc01a8a9cd738e2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.830527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85df55db57bb486f8cc01a8a9cd738e2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.836160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.838242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85df55db57bb486f8cc01a8a9cd738e2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:42.845422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 992/1987] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:42 2022] DELETE /identity/v3/users/85df55db57bb486f8cc01a8a9cd738e2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:42.878114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:42.879900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6caf55-c49b-4731-bfed-1226ad63b609 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:42.890447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 996/1988] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:42.912195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:42.914265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:42.955582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:42.996498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c66a8-3021-4c3c-8b7f-f0c55b013e6f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.002331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.003054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.003340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.003614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] PATH_INFO: `/v3/users/1e38740579694e388b1f17b42b946bf7` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.005442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e38740579694e388b1f17b42b946bf7)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.007200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.068692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e38740579694e388b1f17b42b946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.076194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e38740579694e388b1f17b42b946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.076860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e38740579694e388b1f17b42b946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.089734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e38740579694e388b1f17b42b946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.098006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e38740579694e388b1f17b42b946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.107186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 993/1989] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:42 2022] DELETE /identity/v3/users/1e38740579694e388b1f17b42b946bf7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:43.111775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.112165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.112572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.134607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:43.135915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.151494 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.152982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.212258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 997/1990] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:43.227505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.228873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.229798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343eae1-40db-41dd-8c48-022a1d25a9ae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.235257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.240227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.240227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.240227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] PATH_INFO: `/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.240860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=359f7a136f5046b99fe9ad4a7498a1aa)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.242782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.287709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca4d37-468c-4e78-affb-d149c7654638 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.290067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.291534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359f7a136f5046b99fe9ad4a7498a1aa', 'request_id': 'req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.301458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359f7a136f5046b99fe9ad4a7498a1aa', 'request_id': 'req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.308285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359f7a136f5046b99fe9ad4a7498a1aa', 'request_id': 'req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.318856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] The token cache is being invalidate because project 359f7a136f5046b99fe9ad4a7498a1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:43.319173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.319852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-20dba2ea-1c85-4ffb-91be-7bb3e857f59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.321367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 994/1991] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:43 2022] DELETE /identity/v3/projects/359f7a136f5046b99fe9ad4a7498a1aa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:43.326059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.326338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.326611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.341650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.342266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.342545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.342815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.344645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.347805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:43.348985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.380349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.411242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.412588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce035e01-876d-4bf2-b135-d8387cda30e6 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.422648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 998/1992] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:57:43.427701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 995/1993] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:43.435651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.435941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.436211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.455913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:43.457125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.465030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 999/1994] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:43.469867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.480451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93c8a-347e-4cd2-9d9d-7cb6b8208013 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.486446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.487270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.487656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.487940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.488617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:43.488920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.490683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.503643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 996/1995] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:57:43.507998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.517564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929327c-65ad-403a-abab-dea02a4319ac None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.523214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.523820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.524098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.524368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.524998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.526931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.558683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1000/1996] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:43.559920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.571208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48e819-fbd1-447b-a53a-939046fa3464 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.631391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.632000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.632286 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.632556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.633184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.635411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.662928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 997/1997] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:43.664073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.674455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c2bf4-9277-495a-9fdf-35a0ed2b0049 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.679844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.680596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.680904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.681197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.682186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.684102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.700255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1001/1998] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:43.702686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.702812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.704547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.714278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874f5f5-4911-4070-8602-d880c7bb88f4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.720969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fc6ad-8d60-434a-b2ee-76065742795c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.725794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.726460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.726803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.727068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] PATH_INFO: `/v3/projects/4a5183095ea940b49083d47f986be096` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.728723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5183095ea940b49083d47f986be096)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.730823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.735367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.736168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.736536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.736895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.738246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.741118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.763457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 998/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:43.768572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.782721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7c698c-a5f3-419e-a737-f1f3de72316f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.793306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5183095ea940b49083d47f986be096', 'request_id': 'req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.799664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5183095ea940b49083d47f986be096', 'request_id': 'req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.808010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5183095ea940b49083d47f986be096', 'request_id': 'req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.834180 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] The token cache is being invalidate because project 4a5183095ea940b49083d47f986be096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:43.837035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.837321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.837848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:43.841843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1002/2000] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:43 2022] DELETE /identity/v3/projects/4a5183095ea940b49083d47f986be096 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:43.845584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:43.845584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.862360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.863068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.863681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.864003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] PATH_INFO: `/v3/projects/1f791afc994346b0b30d054cf324ac72/users/b1d61fa002014392811b83187b1d6831/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.886480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f791afc994346b0b30d054cf324ac72, user_id=b1d61fa002014392811b83187b1d6831, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.891081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd189c05-4980-4e37-89f6-d72555df04ab admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:43.908633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 999/2001] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:43 2022] PUT /identity/v3/projects/1f791afc994346b0b30d054cf324ac72/users/b1d61fa002014392811b83187b1d6831/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:43.910272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eef3f-6f5c-4d79-9671-c517828543e5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.910826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eef3f-6f5c-4d79-9671-c517828543e5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.911287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eef3f-6f5c-4d79-9671-c517828543e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.931746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2c8eef3f-6f5c-4d79-9671-c517828543e5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:43.932968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2c8eef3f-6f5c-4d79-9671-c517828543e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:43.933370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0fb7-6c28-4c09-863e-b6bb474205e4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:43.951805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:43.992910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:43.993569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.993874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.994144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] PATH_INFO: `/v3/users/d39bab1f36d24e008120ba7f066f6241` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.994782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1000/2002] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:43.995586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39bab1f36d24e008120ba7f066f6241)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:43.996007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830587b-cc3a-42d5-9d14-f0046794e36b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:43.996292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830587b-cc3a-42d5-9d14-f0046794e36b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:43.997095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830587b-cc3a-42d5-9d14-f0046794e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:43.998047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.027765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8830587b-cc3a-42d5-9d14-f0046794e36b None None] MFA Rules not processed for user `b1d61fa002014392811b83187b1d6831`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.029038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8830587b-cc3a-42d5-9d14-f0046794e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.048831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39bab1f36d24e008120ba7f066f6241'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:44.056632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39bab1f36d24e008120ba7f066f6241'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:44.057221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39bab1f36d24e008120ba7f066f6241'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:44.069599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39bab1f36d24e008120ba7f066f6241'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:44.087463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-462c66c8-06f3-4b5b-998c-0db31948d867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39bab1f36d24e008120ba7f066f6241'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:44.098231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1003/2003] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:43 2022] DELETE /identity/v3/users/d39bab1f36d24e008120ba7f066f6241 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:44.099371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.100218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.100218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.105942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1001/2004] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:44.107492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0502-6a98-440b-8967-360437642541 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.107776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0502-6a98-440b-8967-360437642541 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.108050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd0502-6a98-440b-8967-360437642541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.126517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.127831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.128300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f8fd0502-6a98-440b-8967-360437642541 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.129490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f8fd0502-6a98-440b-8967-360437642541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.187119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.195113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1004/2005] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:44.196041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.199511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.203434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1002/2006] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:44.204277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd0502-6a98-440b-8967-360437642541 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.213639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd0502-6a98-440b-8967-360437642541 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.213804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15dd35-39eb-4cbe-92a5-d4b8f216cc52 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.219075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.219685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.219962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.220141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.220328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.220757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.220867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.221039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.221039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.221039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:44.222219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.222824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.223841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.242926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1003/2007] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:44.244279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.247751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.266663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1005/2008] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:44.266663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.284290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.314100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac256b-9178-46af-ad0e-c4b45353244a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.331222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2cc29-7e78-4729-a54b-7b5b0a3021e7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.336238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.336880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.337157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.337430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.338424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.339997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.363529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.364129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.364410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.364677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.371072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1006/2009] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:44.371072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.374203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.375539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.384856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1004/2010] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:44.385602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.395255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bbfa0-e3be-4d90-ab64-8137337625af service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.434452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dd247-6ca0-48bc-9818-a05926a741dd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.442283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.442879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.443624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.443624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.444170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.446305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.473314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1007/2011] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:44.477301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.478102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.478256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.478555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.479673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.479673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.481828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.488162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73702491-9599-480f-8474-c26c0fcbf037 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.494090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.495236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.495521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.495792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.496505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.498648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.512705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1005/2012] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:44.513015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1008/2013] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:44.516242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.516684 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.536242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9c57-3090-4b8a-b037-e82b95805ae0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.536828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6b646-5a67-4884-8e3c-dc22196be486 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.551896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.552341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.552836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.552960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.553245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.553512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.553615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.553802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.554161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.554500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.556270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.556746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.571017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1006/2014] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:44.571240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1009/2015] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:44.575620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.576190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.587162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96dca9c-d70f-49be-9a0e-6bef4e6e2704 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.588353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6277c-24e4-4603-b4ab-e7cc04c05b60 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.593331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.593962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.594372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.594477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.594650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.595473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.595720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.595986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.596262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] PATH_INFO: `/v3/projects/6e8ba5771a964aa186b7b420482cd52d/users/653d2af7d9b940738c8de6a50fc63ada/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.597103 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.614161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1007/2016] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:44.618468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.621239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8ba5771a964aa186b7b420482cd52d, user_id=653d2af7d9b940738c8de6a50fc63ada, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.628281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1eb68-ba2f-4dab-836e-7e559a5c699a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.633005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d5c20-d0fe-4d46-aa6e-7597a00cb0ea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.640316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.640749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.641030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.641188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] PATH_INFO: `/v3/projects/c00730b67ae04f58be1322a80cc6a4c2/users/ff0c1d9939374d14bbc7cacad41162ea/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.645815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.646100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1010/2017] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:44 2022] PUT /identity/v3/projects/6e8ba5771a964aa186b7b420482cd52d/users/653d2af7d9b940738c8de6a50fc63ada/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:44.658384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.658955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.659477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.669218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00730b67ae04f58be1322a80cc6a4c2, user_id=ff0c1d9939374d14bbc7cacad41162ea, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.671243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210ebc8-6073-493b-b51c-14fd57a20e6a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.691075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1008/2018] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:44 2022] PUT /identity/v3/projects/c00730b67ae04f58be1322a80cc6a4c2/users/ff0c1d9939374d14bbc7cacad41162ea/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:44.696702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.696944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.697199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.698843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] MFA Rules not processed for user `653d2af7d9b940738c8de6a50fc63ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.700655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.718863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:44.720459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.741730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:44.788643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1011/2019] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:44.788643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.789012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.798073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1009/2020] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:44.798958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.809897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178e2aa-7791-410a-a288-71996d33fa43 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.815933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.816559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.816838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.817119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.817756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.819736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:44.853359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1010/2021] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:44 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:44.854511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:44.858462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:44.877683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c24d9-e60b-4400-a957-e0f14ac49bbb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.942462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391cd08-7877-4ce5-bb43-7f509f98fd74 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:44.946548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.947314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.947463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.947741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.973647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:44.974636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:44.975001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:44.975373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] PATH_INFO: `/v3/projects/4cd933e2a6414ec99b51d46aa131d595` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:44.976725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd933e2a6414ec99b51d46aa131d595)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:44.978819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.059844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd933e2a6414ec99b51d46aa131d595', 'request_id': 'req-57e8e319-cd78-48be-a4dd-80b34ce26968'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:45.072957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd933e2a6414ec99b51d46aa131d595', 'request_id': 'req-57e8e319-cd78-48be-a4dd-80b34ce26968'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:45.076353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.079356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.082532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd933e2a6414ec99b51d46aa131d595', 'request_id': 'req-57e8e319-cd78-48be-a4dd-80b34ce26968'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:45.099100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1011/2022] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:45.101422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.103669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] The token cache is being invalidate because project 4cd933e2a6414ec99b51d46aa131d595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:45.103669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:45.109560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-57e8e319-cd78-48be-a4dd-80b34ce26968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:45.111950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1012/2023] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:44 2022] DELETE /identity/v3/projects/4cd933e2a6414ec99b51d46aa131d595 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:45.119678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476674f-8441-496d-8f22-6e4ac2c49aa7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.120362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476674f-8441-496d-8f22-6e4ac2c49aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.120844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476674f-8441-496d-8f22-6e4ac2c49aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.124396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22de20f-1aae-47af-9fd0-9fa9b39cf2c9 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.138031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.157324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a476674f-8441-496d-8f22-6e4ac2c49aa7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.159311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a476674f-8441-496d-8f22-6e4ac2c49aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.182987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.243444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.243730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.244006 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.244308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.245031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.247884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.273448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1012/2024] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:45.277105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.279356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.280340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1013/2025] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:45.292293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.294405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2140653-f20b-4cfc-82cb-444857bb19d4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.295205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.295712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.300071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.300718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.301096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.301954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.302117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.303635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.317300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1013/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:45.321360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.332613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e0ef-1599-4961-858a-6eb997c7b3f6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.337322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.338128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.339224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.339760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.340177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.341154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] PATH_INFO: `/v3/projects/45bbb79d2a9b4429af78434b0d06f60c/users/c7789ca34a6e42f2a40dc5335aeb9caf/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.363197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bbb79d2a9b4429af78434b0d06f60c, user_id=c7789ca34a6e42f2a40dc5335aeb9caf, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.364574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1014/2027] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:45.365564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.366738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.380956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034d68e-425f-4a11-8826-2e8a79f7c1d8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.387163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1014/2028] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:45 2022] PUT /identity/v3/projects/45bbb79d2a9b4429af78434b0d06f60c/users/c7789ca34a6e42f2a40dc5335aeb9caf/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:45.388718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.459815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.460479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.460759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.461029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.461708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:45.462056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.463793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.478404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ac8fa-7a16-414f-a02b-aa5947c84de5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.485327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.486004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.486362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.486663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.487629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.490074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.493225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1015/2029] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:45.497682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.503033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1015/2030] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:45.506760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.513038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070fb67-0cea-46b0-abbf-7cf7a558975c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.519547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6425-3da7-492a-b000-4d104eb73952 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.521041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.521885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.522319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.522868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.523691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.525757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.526221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.526954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.527329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.528562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] PATH_INFO: `/v3/projects/45bbb79d2a9b4429af78434b0d06f60c/users/c7789ca34a6e42f2a40dc5335aeb9caf/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.530871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bbb79d2a9b4429af78434b0d06f60c, user_id=c7789ca34a6e42f2a40dc5335aeb9caf, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.534249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea5f0-f5bd-451b-8312-e922a676561b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.547916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1016/2031] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:45 2022] PUT /identity/v3/projects/45bbb79d2a9b4429af78434b0d06f60c/users/c7789ca34a6e42f2a40dc5335aeb9caf/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:45.560180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.560180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.561651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.563019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1016/2032] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/projects => generated 397 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:45.566450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.594068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] MFA Rules not processed for user `c7789ca34a6e42f2a40dc5335aeb9caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.595392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.640404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea803d-1e37-4fae-a656-d4af41a4cf7f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.641318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.645919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.646688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.646943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.647268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.648044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.650372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.675771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1017/2033] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:45.679595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.686347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1017/2034] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:45.694121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa7888-0dbe-454b-bb3f-2e281ea061f6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.699706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.700509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.700611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.700825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.701162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.701831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.702418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.704158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.707082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:45.722963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1018/2035] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:45.726939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.742246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9da7c-222b-40f2-9e14-5121ec16b67c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.749736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.750272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.750578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.751071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.751656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.754858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.772840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1019/2036] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:45.773036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01221f-d479-4fc5-b3d7-a58aaf0f1a3b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.774125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.776560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.776792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.777038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.777313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.788030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f8ff8-5174-4be6-931c-83262ea2108f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:45.788765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.790093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1060-b9fb-48b3-b8be-070ae02a9418 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.793810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:45.794493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.794859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.795169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] PATH_INFO: `/v3/projects/a1e170f0de53463392be08f7b9f1c023/users/3ca24d4d33534c32900bd4342392f6c9/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.805847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1018/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:45.815860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e170f0de53463392be08f7b9f1c023, user_id=3ca24d4d33534c32900bd4342392f6c9, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:45.818609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b3402-258d-402a-bc69-b50958caa4f3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:45.838286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1020/2038] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:45 2022] PUT /identity/v3/projects/a1e170f0de53463392be08f7b9f1c023/users/3ca24d4d33534c32900bd4342392f6c9/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:45.852234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.852540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.852797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.853994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:45.855268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:45.855851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:45.886134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] MFA Rules not processed for user `ff0c1d9939374d14bbc7cacad41162ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.887916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.890564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] MFA Rules not processed for user `3ca24d4d33534c32900bd4342392f6c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:45.891825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.970050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1021/2039] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:45.971690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:45.973301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:45.986638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1019/2040] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:46.034870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.051933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2e6ed-6d77-4736-9453-f6eeca4a9def None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.055544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.056147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.056424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.056692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.058417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:46.097918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c12ac9-a82e-4b80-a31c-e44dffff6e21 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.101817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.102531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.102819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.103129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.112428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.113858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.122824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1020/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:46.127017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.128656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db071e-fa1d-4205-99a7-ed60de103258 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.138410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1022/2042] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:46.166688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:46.176136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17368127-e939-439f-9a21-806d24a50459 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:46.180450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:46.181552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:46.181552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:46.181743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:46.193927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:46.195412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:46.199377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:46.207510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1021/2043] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:47.579113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493f13-7f3f-43b0-87c4-b73c2deef3e9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:47.579599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493f13-7f3f-43b0-87c4-b73c2deef3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:47.579692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493f13-7f3f-43b0-87c4-b73c2deef3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:47.593700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:47.599881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1a493f13-7f3f-43b0-87c4-b73c2deef3e9 None None] MFA Rules not processed for user `ff0c1d9939374d14bbc7cacad41162ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:47.601209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1a493f13-7f3f-43b0-87c4-b73c2deef3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:47.609963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1023/2044] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:52.538665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.601574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f133a-1e87-4559-9e62-64b826b48551 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.606771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.607425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.607709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.607986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] PATH_INFO: `/v3/users/b1d61fa002014392811b83187b1d6831` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.609224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d61fa002014392811b83187b1d6831)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.611223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.677961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d61fa002014392811b83187b1d6831'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:52.686491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d61fa002014392811b83187b1d6831'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:52.687279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d61fa002014392811b83187b1d6831'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:52.705520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d61fa002014392811b83187b1d6831'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:52.718661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d61fa002014392811b83187b1d6831'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:52.726048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1022/2045] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:52 2022] DELETE /identity/v3/users/b1d61fa002014392811b83187b1d6831 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:52.730528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.731841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.731841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.752718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:52.753968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:52.826759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1024/2046] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:52.843944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:52.845685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:52.937092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791fe5-5b8b-45dc-b879-ae5add52485f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:52.940994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:52.941695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:52.942048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:52.942392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:52.952695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:52.954213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:52.967512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1023/2047] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:53.779807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.781468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:53.812439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df80fcd-fd87-48f4-90a9-bd565156aa2b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.819027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:53.820290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:53.820639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:53.820977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] PATH_INFO: `/v3/projects/1f791afc994346b0b30d054cf324ac72` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:53.822241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f791afc994346b0b30d054cf324ac72)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:53.825986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:53.903783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f791afc994346b0b30d054cf324ac72', 'request_id': 'req-07359dcb-3775-4c44-86f6-c261f99057d8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:53.909478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f791afc994346b0b30d054cf324ac72', 'request_id': 'req-07359dcb-3775-4c44-86f6-c261f99057d8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:53.916353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f791afc994346b0b30d054cf324ac72', 'request_id': 'req-07359dcb-3775-4c44-86f6-c261f99057d8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:53.941503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] The token cache is being invalidate because project 1f791afc994346b0b30d054cf324ac72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:53.941844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:53.942735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:53.946069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1025/2048] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:53 2022] DELETE /identity/v3/projects/1f791afc994346b0b30d054cf324ac72 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:53.951828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:53.962746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def973f1-4749-4df3-8f82-87bed415b7a5 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:53.986105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.035161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.035948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.036375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.036683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] PATH_INFO: `/v3/users/653d2af7d9b940738c8de6a50fc63ada` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.038138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=653d2af7d9b940738c8de6a50fc63ada)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.040264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.101065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653d2af7d9b940738c8de6a50fc63ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.111481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653d2af7d9b940738c8de6a50fc63ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.112267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653d2af7d9b940738c8de6a50fc63ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.126745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653d2af7d9b940738c8de6a50fc63ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.133853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653d2af7d9b940738c8de6a50fc63ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.140480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1024/2049] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:53 2022] DELETE /identity/v3/users/653d2af7d9b940738c8de6a50fc63ada => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:54.414882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.416662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:54.445364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07359dcb-3775-4c44-86f6-c261f99057d8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.521638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.522282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.522564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.522843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] PATH_INFO: `/v3/projects/6e8ba5771a964aa186b7b420482cd52d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.524002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8ba5771a964aa186b7b420482cd52d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.525955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.564485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.588025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8ba5771a964aa186b7b420482cd52d', 'request_id': 'req-fe42baae-05ed-4731-bb99-aa155af79c7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.595566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8ba5771a964aa186b7b420482cd52d', 'request_id': 'req-fe42baae-05ed-4731-bb99-aa155af79c7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.605333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8ba5771a964aa186b7b420482cd52d', 'request_id': 'req-fe42baae-05ed-4731-bb99-aa155af79c7a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.622188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] The token cache is being invalidate because project 6e8ba5771a964aa186b7b420482cd52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:54.622541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.623659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.625670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1026/2050] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:54 2022] DELETE /identity/v3/projects/6e8ba5771a964aa186b7b420482cd52d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:54.629660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.631285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:54.641638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:54.643388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:54.651950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf837-690b-40c2-9b3a-ca0f22a2ba91 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.718126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42baae-05ed-4731-bb99-aa155af79c7a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:54.721506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.722248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.723309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:54.724171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.724461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.724733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] PATH_INFO: `/v3/users/3ca24d4d33534c32900bd4342392f6c9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.725895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca24d4d33534c32900bd4342392f6c9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.727357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.727841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] PATH_INFO: `/v3/users/c7789ca34a6e42f2a40dc5335aeb9caf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.727955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.731463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7789ca34a6e42f2a40dc5335aeb9caf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:54.734517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:54.795666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca24d4d33534c32900bd4342392f6c9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.809495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca24d4d33534c32900bd4342392f6c9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.810088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca24d4d33534c32900bd4342392f6c9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.825699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.830434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca24d4d33534c32900bd4342392f6c9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.839059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7789ca34a6e42f2a40dc5335aeb9caf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.849013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bbe361e7-157c-4ee7-a984-bbbbde96e902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca24d4d33534c32900bd4342392f6c9'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.853916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7789ca34a6e42f2a40dc5335aeb9caf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.855329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7789ca34a6e42f2a40dc5335aeb9caf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.857919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1027/2051] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:54 2022] DELETE /identity/v3/users/3ca24d4d33534c32900bd4342392f6c9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:54.874061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7789ca34a6e42f2a40dc5335aeb9caf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.874817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:54.875895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:54.876956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:54.882412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7789ca34a6e42f2a40dc5335aeb9caf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:54.890944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1025/2052] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:54 2022] DELETE /identity/v3/users/c7789ca34a6e42f2a40dc5335aeb9caf => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:54.901427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:54.902792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:54.949371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:54.994662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1028/2053] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:55.024185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.027559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.144111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eccac5-777e-4f3c-a2f5-9d4df6e47ff6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.148373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.148990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.149268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.149544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.161561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.163050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.177851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1026/2054] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:55.514018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.515687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.525973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.527543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.548348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed19d9-9644-45cc-a047-2e9bde73c99b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.549591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ca190-425d-49e9-a1e7-767a2a067cb0 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.551950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.552743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.553358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.553358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] PATH_INFO: `/v3/projects/45bbb79d2a9b4429af78434b0d06f60c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.555334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=45bbb79d2a9b4429af78434b0d06f60c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.555598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.556300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.556668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.557062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] PATH_INFO: `/v3/projects/a1e170f0de53463392be08f7b9f1c023` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.557533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.558597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e170f0de53463392be08f7b9f1c023)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.572917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.620254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45bbb79d2a9b4429af78434b0d06f60c', 'request_id': 'req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.627950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45bbb79d2a9b4429af78434b0d06f60c', 'request_id': 'req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.635351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45bbb79d2a9b4429af78434b0d06f60c', 'request_id': 'req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.638002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e170f0de53463392be08f7b9f1c023', 'request_id': 'req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.647137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e170f0de53463392be08f7b9f1c023', 'request_id': 'req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.650328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] The token cache is being invalidate because project 45bbb79d2a9b4429af78434b0d06f60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:55.650792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.651588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-887ecee3-bc31-450f-9e6f-a4cf66aa3a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.655921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1029/2055] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:55 2022] DELETE /identity/v3/projects/45bbb79d2a9b4429af78434b0d06f60c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:55.656067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e170f0de53463392be08f7b9f1c023', 'request_id': 'req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.661369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.661782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.662223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.666820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] The token cache is being invalidate because project a1e170f0de53463392be08f7b9f1c023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:55.667141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.667787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87ee53c1-3d2e-4334-bf4e-913e05d70ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:55.670169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1027/2056] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:55 2022] DELETE /identity/v3/projects/a1e170f0de53463392be08f7b9f1c023 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:55.687440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:55.689009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.770991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1030/2057] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:55.780525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.780809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.781211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.803248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:55.804473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:55.813497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1028/2058] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:55.818052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.830485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e57498-6723-48dd-833b-f75d63e21143 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.837465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.838380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.838672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.838927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.839631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:55.839961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.841644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.855141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1031/2059] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:55.861094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.872145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bb314-df54-4e16-ba34-a41fc3013faa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.877112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.877737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.878013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.878287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.879001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.880964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:55.908857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1029/2060] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:55.913068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:55.924208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb043b9-0f35-4763-8116-460fdd67377c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:55.968569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:55.986021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:55.986633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:55.986910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:55.987212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:55.988002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:55.989930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.012908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1032/2061] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:55 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:56.016975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.029066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cae11-3ac7-42f4-a61c-0fe63bbd3721 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.034012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.034614 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.034893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.035211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.035857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.037507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.048230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1030/2062] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:56.052254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.062497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890b82b-bcf9-48c6-9f01-19891b974bc9 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.067187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.067785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.068064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.068338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.068979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.070574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4080b2c-a955-41bb-adff-5d8f4760bf87 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.081392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1033/2063] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:56.085575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.092530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.096346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434474f4-8da7-429b-9db5-672ea6844cde admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.102292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.103133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.103577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.103903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] PATH_INFO: `/v3/projects/5b979e0e638e45158ac3c4d098c556fd/users/50ee36c8b1794cefb7a86b30df28990f/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.119095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b979e0e638e45158ac3c4d098c556fd, user_id=50ee36c8b1794cefb7a86b30df28990f, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.121769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.135076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1031/2064] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:56 2022] PUT /identity/v3/projects/5b979e0e638e45158ac3c4d098c556fd/users/50ee36c8b1794cefb7a86b30df28990f/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:56.146072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.146356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.146628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.178257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] MFA Rules not processed for user `50ee36c8b1794cefb7a86b30df28990f`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:56.179487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.241576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1034/2065] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:56.253169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.254952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.319785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de38aba-27a7-4d62-97d4-0179567b95ad admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.339890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.341169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.341648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.341648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.356369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.358082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891fdf5-3bd2-4bff-b8b9-bd8cec8cba2f service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.368072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1032/2066] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:56.390261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.467988 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab1177-dec2-4e47-b6b0-bd88a65ac277 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.474515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.475186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.475469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.475747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] PATH_INFO: `/v3/users/50ee36c8b1794cefb7a86b30df28990f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.476891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ee36c8b1794cefb7a86b30df28990f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.478810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.546669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ee36c8b1794cefb7a86b30df28990f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:56.552912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:56.555629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ee36c8b1794cefb7a86b30df28990f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:56.555952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ee36c8b1794cefb7a86b30df28990f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:56.566650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ee36c8b1794cefb7a86b30df28990f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:56.590727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ee36c8b1794cefb7a86b30df28990f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:56.608524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1035/2067] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:56 2022] DELETE /identity/v3/users/50ee36c8b1794cefb7a86b30df28990f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:56.612997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.613283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.613558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.633711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:56.634992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.694963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1033/2068] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:56.705312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:56.706834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:56.766536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c292b-f711-40c0-8106-6d373b812310 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:56.770341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:56.770963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:56.771458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:56.771751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:56.782294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:56.783711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e601343-7e11-4f57-ad0a-72466e8d2fec service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:56.794610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1036/2069] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:57:57.084021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.086036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:57.100448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468aaf74-57bb-41ca-bb12-5b0f6e0b8a91 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.106656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.107341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.107618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.107918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] PATH_INFO: `/v3/projects/5b979e0e638e45158ac3c4d098c556fd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.109134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b979e0e638e45158ac3c4d098c556fd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.111076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.161408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b979e0e638e45158ac3c4d098c556fd', 'request_id': 'req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:57.170010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b979e0e638e45158ac3c4d098c556fd', 'request_id': 'req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:57.177854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b979e0e638e45158ac3c4d098c556fd', 'request_id': 'req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:57.188896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] The token cache is being invalidate because project 5b979e0e638e45158ac3c4d098c556fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:57.189182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:57.189840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f52e447d-c4c6-4411-bbf6-36e2b619c9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:57.191622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1034/2070] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:57 2022] DELETE /identity/v3/projects/5b979e0e638e45158ac3c4d098c556fd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:57.196390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.196904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.197001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.216524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:57.217740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:57.284277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1037/2071] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:57.295448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.295734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.296072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.310790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.316457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:57.318311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:57.327793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1035/2072] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:57.332069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.343049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d557d8f-2305-4af1-b1d2-102579a4f1ad None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.347867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.348479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.348758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.349029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.349696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:57.350000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.351652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.367697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1038/2073] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:57.368959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.379722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadda202-7f00-42d8-9570-c9b1476d9186 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.384332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.384923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.385197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.385486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.386104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.388094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.421404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1036/2074] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:57.425675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.437514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104024f-237a-4851-babb-e7cf36894848 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.526050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.526657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.526934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.528334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.528463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.530404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.540210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:57.561089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1039/2075] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/users => generated 280 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:57.562352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.575104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97edbe-b243-40d9-a1b6-748965d25af7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.581211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.581718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.582059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.582400 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.583180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.584938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.611733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1037/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:57.613209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.629712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e931f-ecf3-445a-931e-6cd1626eb037 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.637453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.639355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.640023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.640630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.641700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.643657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80d7ad-9808-4078-b8ea-167d3fc9e576 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.664670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.671030 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1040/2077] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:57.676612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468303e8-f110-4591-9a2f-faf4646584cc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:57.682060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:57.682753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.683058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.683346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] PATH_INFO: `/v3/projects/7a6e59c4c31e4f0984e3710de215267d/users/50cbb8fa7cdc4fcbba298399df3e2f67/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.713221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6e59c4c31e4f0984e3710de215267d, user_id=50cbb8fa7cdc4fcbba298399df3e2f67, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:57.718458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:57.756747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:57.757127 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:57.757555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:57.760440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1038/2078] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:57 2022] PUT /identity/v3/projects/7a6e59c4c31e4f0984e3710de215267d/users/50cbb8fa7cdc4fcbba298399df3e2f67/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:57.822825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] MFA Rules not processed for user `50cbb8fa7cdc4fcbba298399df3e2f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:57.825187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:57.917244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1041/2079] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:57.957248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:57.959063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:58.029215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb368b-4569-448a-87ae-3a8a99f3bc60 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.032892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.033496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.033778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.034057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.045039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.046488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269fda-0517-47df-886f-d1e6917bc54c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.061928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1039/2080] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:58.077990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.153459 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bbf43-0a1f-4192-9009-19168d8637a5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.159614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.160248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.160593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.160873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] PATH_INFO: `/v3/users/50cbb8fa7cdc4fcbba298399df3e2f67` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.162028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cbb8fa7cdc4fcbba298399df3e2f67)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.164008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.235397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cbb8fa7cdc4fcbba298399df3e2f67'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:58.244948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cbb8fa7cdc4fcbba298399df3e2f67'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:58.245873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cbb8fa7cdc4fcbba298399df3e2f67'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:58.258070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cbb8fa7cdc4fcbba298399df3e2f67'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:58.269596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:58.278077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cbb8fa7cdc4fcbba298399df3e2f67'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:58.290019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1042/2081] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:57:58 2022] DELETE /identity/v3/users/50cbb8fa7cdc4fcbba298399df3e2f67 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:58.294729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.295240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.295338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.315083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:58.316368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:58.385735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1040/2082] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:58.399002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:58.400593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:58.471680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267abcb-c72d-4098-a3df-d71fe73555ca admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:58.476759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:58.477035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:58.477239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:58.477510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:58.489176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:58.490682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e768b2e-77a2-4435-a94a-1aac7ba2024a service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:58.502114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1043/2083] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:57:59.057789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.061570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.097442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f2e77-82da-4abb-a7f8-bbbc4ea78bbd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.111718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.112393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.112758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.113096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] PATH_INFO: `/v3/projects/7a6e59c4c31e4f0984e3710de215267d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.114442 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a6e59c4c31e4f0984e3710de215267d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.116522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.179390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:59.195597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6e59c4c31e4f0984e3710de215267d', 'request_id': 'req-b6cdd27d-4d22-4be4-879d-403aa5909430'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:59.206029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6e59c4c31e4f0984e3710de215267d', 'request_id': 'req-b6cdd27d-4d22-4be4-879d-403aa5909430'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:59.214550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6e59c4c31e4f0984e3710de215267d', 'request_id': 'req-b6cdd27d-4d22-4be4-879d-403aa5909430'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:59.243873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] The token cache is being invalidate because project 7a6e59c4c31e4f0984e3710de215267d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:57:59.244390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:59.245043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b6cdd27d-4d22-4be4-879d-403aa5909430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:57:59.247639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1041/2084] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:57:59 2022] DELETE /identity/v3/projects/7a6e59c4c31e4f0984e3710de215267d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:57:59.252530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.252974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.253109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.274045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:59.275263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.336193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1044/2085] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:59.345549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.346062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.346176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.368833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:59.370043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.379455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1042/2086] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:57:59.383872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.394142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecb6f3-3eb6-4985-a45f-e2f722b5ed38 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.398675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.399316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.399592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.399860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.400505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:57:59.400804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.403089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.414937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1045/2087] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:57:59.420459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.429505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3d299-8ce4-4ea1-a5cd-9b3e046e0871 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.435509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.436236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.436384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.436652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.437281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.439275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.471044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1043/2088] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:57:59.473181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.484999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f6cc-f314-4a04-aa75-8dff19d18ca4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.549851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.550462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.550743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.551693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.551693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.553627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.585665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1046/2089] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:57:59.589376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.604597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f638850-960d-419f-b5ba-bcfc8eec7478 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.611716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.613208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.613509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.613921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.614583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.616283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.631012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1044/2090] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:57:59.632885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.647012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bb672-5753-43d1-8012-a003ca1e381d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.652378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.653198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.654191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.654191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.654545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.659571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087fc2a-b7e7-442c-8931-a4ad5968e391 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.662107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:57:59.677702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1047/2091] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:57:59.678611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.690559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a052f-d98d-4659-b9c0-625fb0372a8b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.700873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.701629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.702081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.702360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] PATH_INFO: `/v3/projects/f15d02a0d9064d23beae908af7731122/users/e24b222fb82447bfaa53d0cef95fff03/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.724203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15d02a0d9064d23beae908af7731122, user_id=e24b222fb82447bfaa53d0cef95fff03, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:57:59.726991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:57:59.759235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1045/2092] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:57:59 2022] PUT /identity/v3/projects/f15d02a0d9064d23beae908af7731122/users/e24b222fb82447bfaa53d0cef95fff03/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:57:59.763239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.763523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.763793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:57:59.800895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] MFA Rules not processed for user `e24b222fb82447bfaa53d0cef95fff03`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:57:59.802427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.899579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1048/2093] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:57:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:57:59.911636 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:57:59.913923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:57:59.988594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b6d9f-ae51-46f1-9a09-e33a50c0becb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:57:59.992574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:57:59.993188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:57:59.993685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:57:59.994015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.003567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.005082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.016254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1046/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:57:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:00.051512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.145677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c055c83-9bc9-4a33-a523-02b3ce2082f4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.150404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.151197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.151510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.151780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] PATH_INFO: `/v3/users/e24b222fb82447bfaa53d0cef95fff03` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.154170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24b222fb82447bfaa53d0cef95fff03)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.156069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.220948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24b222fb82447bfaa53d0cef95fff03'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.226087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.227617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.229421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24b222fb82447bfaa53d0cef95fff03'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.230123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24b222fb82447bfaa53d0cef95fff03'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.239914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24b222fb82447bfaa53d0cef95fff03'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.250739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdae10a-994e-4de0-985f-4ccd1da2a2b1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.254079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-fb344b6a-982b-4410-991c-e3930fc35124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24b222fb82447bfaa53d0cef95fff03'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.263418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1049/2095] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:00 2022] DELETE /identity/v3/users/e24b222fb82447bfaa53d0cef95fff03 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:00.267488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.268396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.268684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.270297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.271180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.271703 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.272122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] PATH_INFO: `/v3/users/e18740d304014f8caa523b5d41500140` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.274046 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18740d304014f8caa523b5d41500140)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.278564 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.283357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:00.290124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:00.292740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.337961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18740d304014f8caa523b5d41500140'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.350840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18740d304014f8caa523b5d41500140'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.352306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18740d304014f8caa523b5d41500140'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.372516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:00.379475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18740d304014f8caa523b5d41500140'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.388268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18740d304014f8caa523b5d41500140'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.391636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1050/2096] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:00.404616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1047/2097] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:00 2022] DELETE /identity/v3/users/e18740d304014f8caa523b5d41500140 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:00.404807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.406402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.480320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51048b7-0902-4b0a-93d8-5a54926c319f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.537610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.538335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.538700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.539103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.617697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.619179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.619302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.620784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acf3c7-b809-4346-b893-886125364e93 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.632453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1051/2098] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:00.642956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60d22f-177b-4521-919a-fd766697ec4e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.648360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.648981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.649458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.661552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] PATH_INFO: `/v3/projects/dc7eada23fab4b428e8dd7be1738cea5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.663272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7eada23fab4b428e8dd7be1738cea5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.665336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.716989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7eada23fab4b428e8dd7be1738cea5', 'request_id': 'req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.725357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7eada23fab4b428e8dd7be1738cea5', 'request_id': 'req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.734216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7eada23fab4b428e8dd7be1738cea5', 'request_id': 'req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.745770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] The token cache is being invalidate because project dc7eada23fab4b428e8dd7be1738cea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:00.746077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.746730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a09602c-48a3-4088-941c-e1f0c9ba7b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.748597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1048/2099] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:00 2022] DELETE /identity/v3/projects/dc7eada23fab4b428e8dd7be1738cea5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:00.753577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.753867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.754141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.774495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:00.775796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.795680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:00.848845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1052/2100] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:00.856767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.857056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.857331 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.878238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:00.879480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.888395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1049/2101] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:00.892604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.904464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d72b-2548-43e4-a0d0-ba86beea150b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.909467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.910133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.910355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.910661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.911312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:00.911681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.913315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.913520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.914746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:00.927135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9210087-11ec-4ce3-9d21-81f8d70be307 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.931725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.932343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.932621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.932891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] PATH_INFO: `/v3/projects/f15d02a0d9064d23beae908af7731122` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.934009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15d02a0d9064d23beae908af7731122)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.935941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.940234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1053/2102] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:00.942002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:00.956344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39c235-e59b-477c-a6ed-819af0ec89eb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:00.961211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:00.961803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:00.962082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:00.962352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:00.963029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:00.966261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:00.989179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15d02a0d9064d23beae908af7731122', 'request_id': 'req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:00.996956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1054/2103] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:01.000095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.000297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15d02a0d9064d23beae908af7731122', 'request_id': 'req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:01.010095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15d02a0d9064d23beae908af7731122', 'request_id': 'req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:01.039094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] The token cache is being invalidate because project f15d02a0d9064d23beae908af7731122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:01.039691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:01.040402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:01.042466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1050/2104] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:00 2022] DELETE /identity/v3/projects/f15d02a0d9064d23beae908af7731122 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:01.096077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325b444-9a5b-4458-9ac3-be91d63f580d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.101433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.102374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.102807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.103470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.104885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.108006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.139114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1055/2105] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:00 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:01.139713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.142806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.208986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.243659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e58404-3a44-4fd3-bb98-0f7bb8d203b9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.251511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.252464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.252908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.253176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.253867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.256333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.269164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1051/2106] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:01.273297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.283991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d263-d89a-4216-8582-c0a98a3f2901 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.289659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.290715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.290715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.291234 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.291990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.294241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f73235-83b1-4e1e-9fe1-7a842099dc71 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.305212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1056/2107] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:01.310249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.320599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087497d0-a929-416a-8046-69effe4cb75d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.325911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.326563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.326838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.327142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] PATH_INFO: `/v3/projects/23512a16f5ea4bda8b88759e032fa44b/users/d6b3d98784e1470b9805739a167206f0/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.344284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23512a16f5ea4bda8b88759e032fa44b, user_id=d6b3d98784e1470b9805739a167206f0, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.347127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.363846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1052/2108] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:01 2022] PUT /identity/v3/projects/23512a16f5ea4bda8b88759e032fa44b/users/d6b3d98784e1470b9805739a167206f0/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:01.375520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.375981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.376302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.411368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] MFA Rules not processed for user `d6b3d98784e1470b9805739a167206f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:01.412861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.464637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:01.495834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1057/2109] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:01.507215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:01.508896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:01.567995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b32ad3-ba2d-41c4-9a4d-9af2b4700fd8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:01.571856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:01.572465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:01.572780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:01.573018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:01.583291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:01.584729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7719da-8226-492f-aa8a-16f228f6bec3 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:01.600113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1053/2110] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:10.109690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:10.174155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df6dcb-189d-45d4-adb9-8e3e40347b8a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:10.180725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:10.181353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.181627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.181889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] PATH_INFO: `/v3/users/d6b3d98784e1470b9805739a167206f0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.183021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b3d98784e1470b9805739a167206f0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:10.185518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:10.232681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b3d98784e1470b9805739a167206f0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.239810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b3d98784e1470b9805739a167206f0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.240604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b3d98784e1470b9805739a167206f0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.248924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b3d98784e1470b9805739a167206f0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.262205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b3d98784e1470b9805739a167206f0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.268672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1058/2111] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:10 2022] DELETE /identity/v3/users/d6b3d98784e1470b9805739a167206f0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:10.272954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.273432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.273681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.294859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:10.296086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.354093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1054/2112] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:10.365475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:10.367403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.432532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec741217-c4be-49b9-a6d0-88c01b130d0e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:10.436778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:10.438131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.438131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.438131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.451480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:10.452936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ea4f-9d32-4e9a-b395-e491e201d716 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:10.462771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1059/2113] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:10.825132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:10.826700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.848833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f81f95-81b1-4ab6-9958-fca0f7a37790 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:10.853751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:10.854327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.854608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.854878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] PATH_INFO: `/v3/projects/23512a16f5ea4bda8b88759e032fa44b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.856104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23512a16f5ea4bda8b88759e032fa44b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:10.858100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:10.905686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23512a16f5ea4bda8b88759e032fa44b', 'request_id': 'req-717445b7-793a-48cb-b5eb-c66fdf32b9f9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.907248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:10.913376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23512a16f5ea4bda8b88759e032fa44b', 'request_id': 'req-717445b7-793a-48cb-b5eb-c66fdf32b9f9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.920795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23512a16f5ea4bda8b88759e032fa44b', 'request_id': 'req-717445b7-793a-48cb-b5eb-c66fdf32b9f9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.930999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] The token cache is being invalidate because project 23512a16f5ea4bda8b88759e032fa44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:10.931299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.931932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717445b7-793a-48cb-b5eb-c66fdf32b9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:10.933449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1055/2114] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:10 2022] DELETE /identity/v3/projects/23512a16f5ea4bda8b88759e032fa44b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:10.939062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:10.939346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:10.939615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:10.958950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:10.960155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:10.964220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:11.023121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1060/2115] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:11.032221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.032506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.032782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.052499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:11.053728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:11.062424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1056/2116] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:11.066471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.076682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f75c1-05a2-497a-815a-094476961a25 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.081175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.081759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.082032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.082297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.082932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:11.083273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.084979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.097177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1061/2117] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:11.101964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.111729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485648bf-6a91-4645-98c7-63c2fe414c1f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.116480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.117084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.117360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.117628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.118248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.120227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.150915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1057/2118] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:11 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:11.155641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.169302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d231d-b6dc-4b17-a735-7df8fbf7e192 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.236133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.236749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.237026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.237295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.238003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.239936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.261974 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1062/2119] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:11 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:11.266686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.277879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfcd23-a25b-4a7f-9652-66c6c7e996bd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.283374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.283978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.284287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.284552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.285182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.287670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.306960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1058/2120] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:11.311934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.323169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8d79-42dc-47b7-90e0-d09ef701bb9e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.328238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.328843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.329124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.329394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.330034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.331721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f027a4-e012-4c13-b375-7dc6d3342aff admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.343260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1063/2121] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:11.347900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.358948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3e47c-93dc-4c76-be16-5e6eac72c28f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.363588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.364229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.364506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.364781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] PATH_INFO: `/v3/projects/f2d5b560c9854725bd8aeb528a231d8b/users/c147cb02daad445a9d48ea515ec463bd/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.379985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d5b560c9854725bd8aeb528a231d8b, user_id=c147cb02daad445a9d48ea515ec463bd, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.387326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.400529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1059/2122] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:11 2022] PUT /identity/v3/projects/f2d5b560c9854725bd8aeb528a231d8b/users/c147cb02daad445a9d48ea515ec463bd/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:11.406134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79975a-31f1-4219-bfe8-ab52551578ed None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.406416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79975a-31f1-4219-bfe8-ab52551578ed None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.406686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79975a-31f1-4219-bfe8-ab52551578ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.436474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4e79975a-31f1-4219-bfe8-ab52551578ed None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:11.437593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4e79975a-31f1-4219-bfe8-ab52551578ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:11.497462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1064/2123] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:11.507601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:11.509287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:11.564456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba447445-03d3-47d1-a7d0-99bb491be348 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:11.568025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:11.568620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:11.568899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:11.569169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:11.577994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:11.579452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e90e43-4fdd-46aa-b549-7c82dfabbf86 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:11.588560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1060/2124] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:12.260144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.260601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.260811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.282666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] MFA Rules not processed for user `ff0c1d9939374d14bbc7cacad41162ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:12.284017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:12.287343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.288150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.288673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.297304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:12.343954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] MFA Rules not processed for user `c147cb02daad445a9d48ea515ec463bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:12.345353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:12.355033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1065/2125] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:12.426778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1061/2126] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:12.428048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.439790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde1fe4-b27b-4a81-8355-9e3d955a3caf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.444841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.445520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.445862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.446213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.451292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.451292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.484934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1066/2127] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:12 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:12.486032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.497825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238bb2-e040-49d9-9eda-4e00d79da98f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.501869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:12.561145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.561755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.562039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.562313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.562933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.564914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.588707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1062/2128] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:12 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:12.591752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.601795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353219c-0c71-4a79-b837-683d8908ab88 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.606578 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.607202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.607483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.607757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.608398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.609992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.622154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1067/2129] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:12.625175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.635132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311b9d6-60c3-4863-a3d0-4a19428a994c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.640391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.641058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.641339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.641612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] PATH_INFO: `/v3/projects/60d1a8f0cac848049c97bc563bb623e1/users/0b7f9f133fb94cf7969c04d9b3f5856c/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.656612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d1a8f0cac848049c97bc563bb623e1, user_id=0b7f9f133fb94cf7969c04d9b3f5856c, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.659322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.673904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1063/2130] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:12 2022] PUT /identity/v3/projects/60d1a8f0cac848049c97bc563bb623e1/users/0b7f9f133fb94cf7969c04d9b3f5856c/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:12.677837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.688455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24544e6b-704d-4d3e-b0cb-f64717658278 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.748344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.749116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.749392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.749659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.750289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.751914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed2709-9553-4796-a1d2-c4f03ec18e9e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.764436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1068/2131] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:12.768847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:12.779244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bcc02-bd7d-475b-b13e-2f31d4540405 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:12.784011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:12.784660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:12.784939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:12.785211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] PATH_INFO: `/v3/projects/60d1a8f0cac848049c97bc563bb623e1/users/0b7f9f133fb94cf7969c04d9b3f5856c/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:12.787179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d1a8f0cac848049c97bc563bb623e1, user_id=0b7f9f133fb94cf7969c04d9b3f5856c, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:12.790034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:12.802419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1064/2132] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:12 2022] PUT /identity/v3/projects/60d1a8f0cac848049c97bc563bb623e1/users/0b7f9f133fb94cf7969c04d9b3f5856c/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:13.716247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:13.716756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:13.716868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:13.745736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] MFA Rules not processed for user `0b7f9f133fb94cf7969c04d9b3f5856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:13.747069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:13.819947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1069/2133] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:13 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:15.123849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:15.192623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d25f-67fc-4adb-8662-bab081dcac95 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:15.198245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:15.198911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:15.199218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:15.200169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] PATH_INFO: `/v3/users/c147cb02daad445a9d48ea515ec463bd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:15.201312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c147cb02daad445a9d48ea515ec463bd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:15.204593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:15.260668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c147cb02daad445a9d48ea515ec463bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.270036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c147cb02daad445a9d48ea515ec463bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.270656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c147cb02daad445a9d48ea515ec463bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.278098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:15.287629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c147cb02daad445a9d48ea515ec463bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.295015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c147cb02daad445a9d48ea515ec463bd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.301692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1065/2134] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:15 2022] DELETE /identity/v3/users/c147cb02daad445a9d48ea515ec463bd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:15.495939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:15.507507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4732699-628f-46ad-a94e-3ec7c4449300 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:15.513580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:15.576841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:15.577471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:15.577754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:15.578027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] PATH_INFO: `/v3/projects/f2d5b560c9854725bd8aeb528a231d8b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:15.580021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d5b560c9854725bd8aeb528a231d8b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:15.582193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:15.632889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d5b560c9854725bd8aeb528a231d8b', 'request_id': 'req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.639417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d5b560c9854725bd8aeb528a231d8b', 'request_id': 'req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.645804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d5b560c9854725bd8aeb528a231d8b', 'request_id': 'req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.659630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] The token cache is being invalidate because project f2d5b560c9854725bd8aeb528a231d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:15.659923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.661347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.666282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1070/2135] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:15 2022] DELETE /identity/v3/projects/f2d5b560c9854725bd8aeb528a231d8b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:15.667704 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:15.669496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:15.685362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641ddd-8ba8-46dc-9e84-9400498e4a3d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:15.765095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:15.765726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:15.766001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:15.766267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] PATH_INFO: `/v3/users/0b7f9f133fb94cf7969c04d9b3f5856c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:15.767431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7f9f133fb94cf7969c04d9b3f5856c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:15.769378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:15.820014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7f9f133fb94cf7969c04d9b3f5856c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.827680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7f9f133fb94cf7969c04d9b3f5856c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.828386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7f9f133fb94cf7969c04d9b3f5856c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.847807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7f9f133fb94cf7969c04d9b3f5856c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.856184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6f8ab6f8-5be9-4ec5-ae60-9179b0f616ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7f9f133fb94cf7969c04d9b3f5856c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:15.862584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1066/2136] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:15 2022] DELETE /identity/v3/users/0b7f9f133fb94cf7969c04d9b3f5856c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:16.045233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.046936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.069210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55f092-cc9f-41d3-b8f1-53818f7ebdb7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.143066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.143720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.143983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.144402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] PATH_INFO: `/v3/projects/60d1a8f0cac848049c97bc563bb623e1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.146657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d1a8f0cac848049c97bc563bb623e1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.148412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.195099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.206520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d1a8f0cac848049c97bc563bb623e1', 'request_id': 'req-bf3e5571-222e-49b3-bf16-e37f363b49cf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:16.214667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d1a8f0cac848049c97bc563bb623e1', 'request_id': 'req-bf3e5571-222e-49b3-bf16-e37f363b49cf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:16.225202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d1a8f0cac848049c97bc563bb623e1', 'request_id': 'req-bf3e5571-222e-49b3-bf16-e37f363b49cf'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:16.241001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] The token cache is being invalidate because project 60d1a8f0cac848049c97bc563bb623e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:16.241284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:16.241970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bf3e5571-222e-49b3-bf16-e37f363b49cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:16.243524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1071/2137] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:16 2022] DELETE /identity/v3/projects/60d1a8f0cac848049c97bc563bb623e1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:16.250720 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.251288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.251288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.270964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:16.272255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.325591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.338254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1067/2138] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:16.348047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.348353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.348624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.375792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:16.377143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.385626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1072/2139] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:16.390362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.400136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb60a0-3067-4002-804d-647841db1a37 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.405472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.406060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.406332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.406596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.407299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:16.407595 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.409174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.421503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1068/2140] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:58:16.425997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.435364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8bdce-68c0-45e5-b7b3-7a355f8fe901 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.440112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.440707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.440985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.441259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.441892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.443837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.474716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1073/2141] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:16.476167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.487223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96b8b5-7e8a-4857-98c1-52f4e7096f2d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.546405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.547111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.547418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.547691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.548326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.550277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.574087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1069/2142] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:16.578788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.591106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581e6d4-030b-4c1f-9849-bcd068f6dd8e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.597199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.598190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.598701 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.599251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.600309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.601971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.613128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1074/2143] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:16.617213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.627695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f624003-4dbc-4c9b-ba41-9c86c500a7eb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.632555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.633156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.633437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.633707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.634351 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.635983 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dbe04-f134-480a-bccb-195ba9f677eb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.648165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1070/2144] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:16.652272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.662734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761af0e-3a98-4c4b-8421-3d47df817197 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.667314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.667952 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.668328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.668610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] PATH_INFO: `/v3/projects/9d4a9e05d664449a82f3fa8719b104b9/users/98005bb1ba18430391060b4bb2c3acaf/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.683931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4a9e05d664449a82f3fa8719b104b9, user_id=98005bb1ba18430391060b4bb2c3acaf, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.686589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.699068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1075/2145] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:16 2022] PUT /identity/v3/projects/9d4a9e05d664449a82f3fa8719b104b9/users/98005bb1ba18430391060b4bb2c3acaf/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:16.709670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.709890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.710158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.738369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] MFA Rules not processed for user `98005bb1ba18430391060b4bb2c3acaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:16.739702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:16.804292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1071/2146] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:16.808783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.872830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571050f-01e8-423a-a7f5-142f67f5415d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.877706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.878305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.878591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.878854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.879511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.881408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:16.912918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1076/2147] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:16.914951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:16.925975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111953a-75cf-4280-b7dd-7fc7c4fba2c7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:16.974120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:16.985399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:16.986010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:16.986378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:16.986657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:16.987313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:16.989226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.018426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1072/2148] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:16 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:17.022414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.033375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcf633-b840-4418-aa16-dbc8be957189 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.038011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.038615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.038896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.039200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.039840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.041433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.052026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1077/2149] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:17.056125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.067637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cd5e8-15f4-414f-b953-2e6cf3b27239 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.074947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.075659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.075920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.076191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] PATH_INFO: `/v3/projects/1eadcfb879434081b715507ad0bf6c38/users/d7d87801866f43d798e79d23ece07f14/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.091771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eadcfb879434081b715507ad0bf6c38, user_id=d7d87801866f43d798e79d23ece07f14, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.094421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.107457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1073/2150] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:17 2022] PUT /identity/v3/projects/1eadcfb879434081b715507ad0bf6c38/users/d7d87801866f43d798e79d23ece07f14/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:17.111785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.122356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e8f55-0b53-4ca0-81f3-4c13660e4dea admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.158926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:17.178295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.178883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.179186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.179454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.180078 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.181645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f5bc0-54ed-4953-bf7f-8fdb8981aeb3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.191735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1078/2151] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:17.195840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.206543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff81e55-02da-48da-8830-27f3e68aa499 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.211028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.211660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.211960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.212227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] PATH_INFO: `/v3/projects/1eadcfb879434081b715507ad0bf6c38/users/d7d87801866f43d798e79d23ece07f14/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.214102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eadcfb879434081b715507ad0bf6c38, user_id=d7d87801866f43d798e79d23ece07f14, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.216822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.227435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1074/2152] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:17 2022] PUT /identity/v3/projects/1eadcfb879434081b715507ad0bf6c38/users/d7d87801866f43d798e79d23ece07f14/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:17.238622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.238914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.239229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.267743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] MFA Rules not processed for user `d7d87801866f43d798e79d23ece07f14`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:17.268945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:17.341332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1079/2153] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:17.352345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.354081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:17.413702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b674dc-9e94-4b72-9f81-38077bc5a924 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.417322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.417908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.418182 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.418492 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.427063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.428478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.437782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1075/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:17.522542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:17.534013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb921aa-e8eb-4d6d-815e-6191a4d1d39c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:17.537787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:17.538383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:17.538661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:17.538933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:17.595734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:17.597170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b341b85-ab3c-4ce5-b430-50edbdfc05b1 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:17.606062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1080/2155] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:27.433483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=46,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:27.602048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=73 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.030187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.092548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e074c48-7f30-42e0-91f0-9b12488be91b service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.097054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.097672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.097949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.098219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] PATH_INFO: `/v3/users/98005bb1ba18430391060b4bb2c3acaf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.099416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98005bb1ba18430391060b4bb2c3acaf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.101334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.147993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98005bb1ba18430391060b4bb2c3acaf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.156669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98005bb1ba18430391060b4bb2c3acaf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.157273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98005bb1ba18430391060b4bb2c3acaf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.169360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98005bb1ba18430391060b4bb2c3acaf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.176173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98005bb1ba18430391060b4bb2c3acaf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.181928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1076/2156] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:28 2022] DELETE /identity/v3/users/98005bb1ba18430391060b4bb2c3acaf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:28.187158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.187746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.187746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.208668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:28.209909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:28.266762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1081/2157] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:28.276551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.278486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:28.335255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c99a9-a6b3-456a-a131-6fe140ffaff1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.338864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.339497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.339778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.340051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.349495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.350938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.361213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1077/2158] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:28.656413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.658126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:28.675737 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd247e0b-995f-47b9-8f4f-85139853a627 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.680513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.681186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.681468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.681742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] PATH_INFO: `/v3/projects/9d4a9e05d664449a82f3fa8719b104b9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.682888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4a9e05d664449a82f3fa8719b104b9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.685412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.736324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4a9e05d664449a82f3fa8719b104b9', 'request_id': 'req-75d643aa-561d-4e38-8717-3ad75732f4ae'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.741682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4a9e05d664449a82f3fa8719b104b9', 'request_id': 'req-75d643aa-561d-4e38-8717-3ad75732f4ae'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.748495 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4a9e05d664449a82f3fa8719b104b9', 'request_id': 'req-75d643aa-561d-4e38-8717-3ad75732f4ae'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.761416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] The token cache is being invalidate because project 9d4a9e05d664449a82f3fa8719b104b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:28.761696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.762319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.763859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1082/2159] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:28 2022] DELETE /identity/v3/projects/9d4a9e05d664449a82f3fa8719b104b9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:28.767969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:28.777668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee673875-cbb0-464d-b281-7f2a98a9d5a1 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:28.797679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:28.850371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:28.851084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:28.851360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:28.852053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] PATH_INFO: `/v3/users/d7d87801866f43d798e79d23ece07f14` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:28.853215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d87801866f43d798e79d23ece07f14)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:28.855162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:28.908952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d87801866f43d798e79d23ece07f14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.916766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d87801866f43d798e79d23ece07f14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.920949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d87801866f43d798e79d23ece07f14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.936697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d87801866f43d798e79d23ece07f14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.944814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0136c1e-5973-446e-87eb-5bff238fa15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d87801866f43d798e79d23ece07f14'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:28.952574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1078/2160] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:28 2022] DELETE /identity/v3/users/d7d87801866f43d798e79d23ece07f14 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:29.251505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.252930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:29.269951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d643aa-561d-4e38-8717-3ad75732f4ae admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.352296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.353149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.353481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.353802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] PATH_INFO: `/v3/projects/1eadcfb879434081b715507ad0bf6c38` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.354957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eadcfb879434081b715507ad0bf6c38)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.357116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.391407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:29.413993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eadcfb879434081b715507ad0bf6c38', 'request_id': 'req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:29.420739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eadcfb879434081b715507ad0bf6c38', 'request_id': 'req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:29.428155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eadcfb879434081b715507ad0bf6c38', 'request_id': 'req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:29.446465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] The token cache is being invalidate because project 1eadcfb879434081b715507ad0bf6c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:29.447415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:29.448194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6a9d83e1-82cc-431f-8cf0-0174eb70edfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:29.450839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1083/2161] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:29 2022] DELETE /identity/v3/projects/1eadcfb879434081b715507ad0bf6c38 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:29.459307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.459824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.460243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.485963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:29.487256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:29.560864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1079/2162] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:29.568415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.568758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.569060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.590149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:29.591956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:29.600010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1084/2163] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:29.605054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.615203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4394b5-d447-4efb-b4bb-d0c61cb82b4b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.619899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.620554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.620830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.621101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.621855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:29.622155 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.624076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.636052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1080/2164] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:29.641044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.650273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61f666-f898-4e81-aade-17ddcb5eeb01 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.655651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.656236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.656519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.657097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.657442 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.659368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.686771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1085/2165] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:29 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:29.691482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.702123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d1cf2-fc2a-412e-b6c5-d6f4467ecb69 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.727444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:29.761121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.761634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.761908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.762745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.763427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.765333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.784947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1081/2166] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:29 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:29.789144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.799465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867e236-5bb5-4968-abe1-8bdeb9cbfddb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.804359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.804945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.805217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.805482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.806139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.808069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.819225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1086/2167] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:29.823313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.834085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6219e-de48-497a-b5a8-2cf3bea3a852 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.838683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.839305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.839584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.839856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.841002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.842583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26335b5-4adc-4c3f-aeef-159b4e2f678d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.852238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1082/2168] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:29.857115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:29.866889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cba923-aa98-4e17-99fb-eb02bd5516f4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:29.871464 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:29.872090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.872393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.872662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] PATH_INFO: `/v3/projects/c03469e46c534717a0ea8fe9839d40aa/users/ae46be0b107b4123850d774d6551a35a/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.886925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03469e46c534717a0ea8fe9839d40aa, user_id=ae46be0b107b4123850d774d6551a35a, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:29.889632 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:29.902150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1087/2169] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:29 2022] PUT /identity/v3/projects/c03469e46c534717a0ea8fe9839d40aa/users/ae46be0b107b4123850d774d6551a35a/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:29.912713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:29.913001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:29.913275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:29.941563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] MFA Rules not processed for user `ae46be0b107b4123850d774d6551a35a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:29.942791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:30.010329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1083/2170] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:30.015434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.086516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d37816-4ef4-4839-9458-57caeec18783 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.091781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.092381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.092663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.092938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.093574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.095548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.124671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1088/2171] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:30 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:30.128950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.140649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748241d-25b8-40db-8934-2b3faa5b0cd7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.200320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.200689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.200971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.201244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.201913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.203917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.234034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1084/2172] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:30 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:30.238165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.248361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314d7e-e4f4-409c-b19c-9255449fb60e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.253306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.253900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.254176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.254467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.255136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.256712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.267922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1089/2173] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:30.271885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.282646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7e12-8681-46d7-a586-fda9b34c565f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.287228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.287862 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.288143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.288417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] PATH_INFO: `/v3/projects/cb3c29d874db4bbeb84158ef99bd076c/users/ae2121d51bf448f19e0e82e929cee9c2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.303026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3c29d874db4bbeb84158ef99bd076c, user_id=ae2121d51bf448f19e0e82e929cee9c2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.305697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.319267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1085/2174] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:30 2022] PUT /identity/v3/projects/cb3c29d874db4bbeb84158ef99bd076c/users/ae2121d51bf448f19e0e82e929cee9c2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:30.323714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.334249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5598ed-e684-473f-a296-8e547a9594a2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.354332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:30.391861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.392453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.392729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.392997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.393624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.395223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e192d1-1475-4f38-819a-95c6487edacd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.405261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1090/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:30.409593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.415117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:30.421655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d338e34-b8fb-4b80-8ce6-e310699ef573 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.426547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.427234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.427513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.427788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] PATH_INFO: `/v3/projects/cb3c29d874db4bbeb84158ef99bd076c/users/ae2121d51bf448f19e0e82e929cee9c2/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.429718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3c29d874db4bbeb84158ef99bd076c, user_id=ae2121d51bf448f19e0e82e929cee9c2, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.432412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.443242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1086/2176] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:30 2022] PUT /identity/v3/projects/cb3c29d874db4bbeb84158ef99bd076c/users/ae2121d51bf448f19e0e82e929cee9c2/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:30.454692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.455029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.455302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.481932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] MFA Rules not processed for user `ae2121d51bf448f19e0e82e929cee9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:30.483155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:30.551385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1091/2177] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:30.563350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.564966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:30.648723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e088d-6d7b-4c6f-bde3-8aa42b576da4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.653953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.654571 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.655194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.655194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.665072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.666522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d3f6f-50f0-434b-8bd4-146d2976927b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.676987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1087/2178] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:58:30.706779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.769664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ef1b5-7a56-408b-b27c-f7aad818ecb4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:30.774066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:30.774675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.774947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.775241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] PATH_INFO: `/v3/users/ae46be0b107b4123850d774d6551a35a` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.776435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae46be0b107b4123850d774d6551a35a)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:30.778811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:30.822492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae46be0b107b4123850d774d6551a35a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:30.823728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:30.831456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae46be0b107b4123850d774d6551a35a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:30.832152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae46be0b107b4123850d774d6551a35a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:30.839628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae46be0b107b4123850d774d6551a35a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:30.852007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae46be0b107b4123850d774d6551a35a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:30.857752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1092/2179] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:30 2022] DELETE /identity/v3/users/ae46be0b107b4123850d774d6551a35a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:30.862223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:30.862778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:30.862778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:30.881833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:30.883092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:30.939955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1088/2180] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:30.950297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:30.951937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:31.011788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600b82d-5b91-41ec-9710-f9f380bc44cc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:31.016997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:31.017534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.017816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.018089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.027375 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:31.029488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:31.039027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1093/2181] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:31.310187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:31.311555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:31.341645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39898c4b-4b43-49df-9138-55f44618f56e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:31.347204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:31.347763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.347951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.348218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] PATH_INFO: `/v3/projects/c03469e46c534717a0ea8fe9839d40aa` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.349330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03469e46c534717a0ea8fe9839d40aa)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:31.351257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:31.399195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03469e46c534717a0ea8fe9839d40aa', 'request_id': 'req-3d729d20-1a7b-4535-9771-4b4535627eb4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.409801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03469e46c534717a0ea8fe9839d40aa', 'request_id': 'req-3d729d20-1a7b-4535-9771-4b4535627eb4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.416421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03469e46c534717a0ea8fe9839d40aa', 'request_id': 'req-3d729d20-1a7b-4535-9771-4b4535627eb4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.428093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] The token cache is being invalidate because project c03469e46c534717a0ea8fe9839d40aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:31.428391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.429020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.430768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1089/2182] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:31 2022] DELETE /identity/v3/projects/c03469e46c534717a0ea8fe9839d40aa => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:31.434554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:31.444623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4cb7-9374-44dd-83e3-0cefa6b85ed3 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:31.500302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:31.500913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:31.501187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:31.501484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] PATH_INFO: `/v3/users/ae2121d51bf448f19e0e82e929cee9c2` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:31.502586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2121d51bf448f19e0e82e929cee9c2)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:31.504529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:31.561593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2121d51bf448f19e0e82e929cee9c2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.575735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2121d51bf448f19e0e82e929cee9c2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.576494 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2121d51bf448f19e0e82e929cee9c2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.584418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2121d51bf448f19e0e82e929cee9c2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.589730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:31.598892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8d2c83ef-bcbb-4964-89b6-5fb1d4a0610d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2121d51bf448f19e0e82e929cee9c2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:31.605252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1094/2183] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:31 2022] DELETE /identity/v3/users/ae2121d51bf448f19e0e82e929cee9c2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:31.955877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:31.957678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:31.970204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d20-1a7b-4535-9771-4b4535627eb4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:31.982134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:32.035189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.035813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.036090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.036359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] PATH_INFO: `/v3/projects/cb3c29d874db4bbeb84158ef99bd076c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.037473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3c29d874db4bbeb84158ef99bd076c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.039424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.085624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3c29d874db4bbeb84158ef99bd076c', 'request_id': 'req-a43ba388-c84a-4338-8644-45fc47aba44c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:32.092874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3c29d874db4bbeb84158ef99bd076c', 'request_id': 'req-a43ba388-c84a-4338-8644-45fc47aba44c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:32.099484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3c29d874db4bbeb84158ef99bd076c', 'request_id': 'req-a43ba388-c84a-4338-8644-45fc47aba44c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:32.111122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] The token cache is being invalidate because project cb3c29d874db4bbeb84158ef99bd076c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:32.111413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:32.112367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a43ba388-c84a-4338-8644-45fc47aba44c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:32.113801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1090/2184] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:31 2022] DELETE /identity/v3/projects/cb3c29d874db4bbeb84158ef99bd076c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:32.118592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.119122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.119214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.140907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:32.142092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:32.204060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1095/2185] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:32.212269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.212559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.212832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.235090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:32.236309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:32.244208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1091/2186] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:32.249278 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.259100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d93bd-5afd-4a92-9ce1-0253b69f856f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.264761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.265422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.265698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.265996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.266712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:32.267044 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.268660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.281166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1096/2187] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:32.285677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.295991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f6d4-076b-4c70-8556-bab81438662b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.306491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.307162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.307505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.307775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.308407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.311253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.340188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1092/2188] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:32.344147 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.354503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af663-0520-42f9-a4cd-913a7b0d4147 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.444475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.445149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.445426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.445707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.446331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.448274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.473513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1097/2189] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:32.477601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.491798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd049a-393a-4197-bdaf-cd523b96bc4b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.498039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.498314 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.499001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.499388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.500600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.502740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.515429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1093/2190] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:32.519368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.534954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda4a6d-9d1f-48a9-bed9-28de89409f27 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.539041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.539711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.540060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.540396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.541112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.542690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfdbe5-6f00-431d-89ea-51b8c4ae47d2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.554272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1098/2191] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:32.558643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.569740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0922305-3793-4495-af20-c618e5a0aa21 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.574534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.575220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.575504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.575810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] PATH_INFO: `/v3/projects/927f876d6c644e3c949fe29e748fe803/users/603e7315383e4bf68e1c34ceb4ca8c0b/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.590273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=927f876d6c644e3c949fe29e748fe803, user_id=603e7315383e4bf68e1c34ceb4ca8c0b, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.592993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.605294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1094/2192] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:32 2022] PUT /identity/v3/projects/927f876d6c644e3c949fe29e748fe803/users/603e7315383e4bf68e1c34ceb4ca8c0b/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:32.616093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.616383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.616705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.644630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] MFA Rules not processed for user `603e7315383e4bf68e1c34ceb4ca8c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:32.645841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:32.698700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:32.724799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1099/2193] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:32.729589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.774343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:32.794414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8189b-7953-4bc5-9591-6dbe41d2d01d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.799058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.799662 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.799941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.800196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.800823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.802725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.828573 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1095/2194] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:32.832621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.843720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78396b77-321d-408d-8e0f-d50a991fc34f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.898316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.898830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.899146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.899420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.900028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.901964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.923587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1100/2195] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:32 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:32.927903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.938258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14583558-9687-4f31-a9ab-cc651e9abad2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.943298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.943898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.944178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.944451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:32.945092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:32.946687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:32.957399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1096/2196] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:32.961670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:32.979697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415835c-cbf5-4261-a748-4d3d534789b2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:32.985642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:32.986293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:32.986569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:32.986839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] PATH_INFO: `/v3/projects/6ab38471172b4ccfbd839eba70a9ba61/users/2a969f00878548679554fc7a1723df6d/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.003940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab38471172b4ccfbd839eba70a9ba61, user_id=2a969f00878548679554fc7a1723df6d, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.006594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.027223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1101/2197] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:32 2022] PUT /identity/v3/projects/6ab38471172b4ccfbd839eba70a9ba61/users/2a969f00878548679554fc7a1723df6d/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:33.031826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.042146 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e42d1-a998-4450-8ac0-c4aac2f5c9a3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.110807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.111444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.111725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.112472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.112626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.114220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6e113-8a52-41e9-bee7-abca9ad25a44 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.126309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1097/2198] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:33.130467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.141752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f60bc5-db34-4357-b18c-665d9df06b5c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.146432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.147117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.147399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.147672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] PATH_INFO: `/v3/projects/6ab38471172b4ccfbd839eba70a9ba61/users/2a969f00878548679554fc7a1723df6d/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.149642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab38471172b4ccfbd839eba70a9ba61, user_id=2a969f00878548679554fc7a1723df6d, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.152305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.163194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1102/2199] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:33 2022] PUT /identity/v3/projects/6ab38471172b4ccfbd839eba70a9ba61/users/2a969f00878548679554fc7a1723df6d/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:33.174831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.175145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.175423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.204049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] MFA Rules not processed for user `2a969f00878548679554fc7a1723df6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:33.205298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:33.293185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1098/2200] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:33.301060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.303635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:33.375832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bc1e3-790d-4bca-a1e0-55583e993aef admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.379794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.380511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.380789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.381079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.391733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.393182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.407313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1103/2201] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:33.830679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.840710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb831c-6287-447f-ae67-dc8e134ca61c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.845498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a969f00878548679554fc7a1723df6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ab38471172b4ccfbd839eba70a9ba61', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.846101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.846384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.846659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.847353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:33.847658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.849275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.861508 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1099/2202] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:33.865913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.876559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdf85e-e846-4573-a582-2642a6aeda59 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.881038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a969f00878548679554fc7a1723df6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ab38471172b4ccfbd839eba70a9ba61', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.881617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.881896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.882165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.882815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.884757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:33.910988 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1104/2203] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:33 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:33.915263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:33.925655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bde20-c9ed-4c9c-8603-e6718ccd662b tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:33.947004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:33.990957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a969f00878548679554fc7a1723df6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ab38471172b4ccfbd839eba70a9ba61', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:33.991680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:33.992011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:33.992383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:33.993186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:33.993491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:33.995134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.008142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1100/2204] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:34.045731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.055897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e970ad1-4f1d-4142-a821-c9225b95b6a2 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.061314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a969f00878548679554fc7a1723df6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ab38471172b4ccfbd839eba70a9ba61', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.061944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.062229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.062508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] PATH_INFO: `/v3/projects/bf1c99b61b5f4409b2920c556f6cd9bb` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.063648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf1c99b61b5f4409b2920c556f6cd9bb)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.065558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.077163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.113567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1c99b61b5f4409b2920c556f6cd9bb', 'request_id': 'req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.121245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1c99b61b5f4409b2920c556f6cd9bb', 'request_id': 'req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.129797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1c99b61b5f4409b2920c556f6cd9bb', 'request_id': 'req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.141762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] The token cache is being invalidate because project bf1c99b61b5f4409b2920c556f6cd9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:34.142357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.142750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f7643db7-6bd8-46a6-893b-f6c5f5f1594c tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.144327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1105/2205] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:34 2022] DELETE /identity/v3/projects/bf1c99b61b5f4409b2920c556f6cd9bb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:34.362863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.364040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:34.447276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a77e80-e341-425c-9b38-c45cd5bf5954 tempest-VolumeQuotaClassesTest-2024045034 tempest-VolumeQuotaClassesTest-2024045034-project-admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.502412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.503071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.503352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.503623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] PATH_INFO: `/v3/users/603e7315383e4bf68e1c34ceb4ca8c0b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.504739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603e7315383e4bf68e1c34ceb4ca8c0b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.506699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.553131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603e7315383e4bf68e1c34ceb4ca8c0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.560020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603e7315383e4bf68e1c34ceb4ca8c0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.560867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603e7315383e4bf68e1c34ceb4ca8c0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.570841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:34.573263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603e7315383e4bf68e1c34ceb4ca8c0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.582655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603e7315383e4bf68e1c34ceb4ca8c0b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:34.588652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1101/2206] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:34 2022] DELETE /identity/v3/users/603e7315383e4bf68e1c34ceb4ca8c0b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:34.593944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.594227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.594501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.613410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:34.614596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:34.675571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1106/2207] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:34.685612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:34.687070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:34.745848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe03f-9af4-4a87-9e26-eabf5c950793 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:34.749861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:34.750461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:34.750734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:34.751027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:34.760036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:34.761453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:34.771347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1102/2208] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:35.062411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:35.064129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:35.089817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a97bc-b040-4ef8-877e-57ab37e55879 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:35.103929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:35.104587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:35.104870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:35.105271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] PATH_INFO: `/v3/projects/927f876d6c644e3c949fe29e748fe803` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:35.106512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927f876d6c644e3c949fe29e748fe803)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:35.108465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:35.172419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927f876d6c644e3c949fe29e748fe803', 'request_id': 'req-41772b40-312a-4355-a70f-25baa906a5d2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.179069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927f876d6c644e3c949fe29e748fe803', 'request_id': 'req-41772b40-312a-4355-a70f-25baa906a5d2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.185742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927f876d6c644e3c949fe29e748fe803', 'request_id': 'req-41772b40-312a-4355-a70f-25baa906a5d2'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.201669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] The token cache is being invalidate because project 927f876d6c644e3c949fe29e748fe803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:35.201956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.202635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.204157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1107/2209] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:35 2022] DELETE /identity/v3/projects/927f876d6c644e3c949fe29e748fe803 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:35.208212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:35.218360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc93b-f8ce-467a-8746-db1effa82da3 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:35.281021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:35.281648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:35.281929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:35.282317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] PATH_INFO: `/v3/users/2a969f00878548679554fc7a1723df6d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:35.283469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a969f00878548679554fc7a1723df6d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:35.285396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:35.334062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a969f00878548679554fc7a1723df6d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.341907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a969f00878548679554fc7a1723df6d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.342549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a969f00878548679554fc7a1723df6d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.355498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a969f00878548679554fc7a1723df6d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.363472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0d7a51b-cebe-4df6-9e0b-5b3ed0c94193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a969f00878548679554fc7a1723df6d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:35.370213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1103/2210] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:35 2022] DELETE /identity/v3/users/2a969f00878548679554fc7a1723df6d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:35.846702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:35.848657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:35.870671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41772b40-312a-4355-a70f-25baa906a5d2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:35.915354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:35.964426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:35.965841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:35.966126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:35.966409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] PATH_INFO: `/v3/projects/6ab38471172b4ccfbd839eba70a9ba61` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:35.967571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab38471172b4ccfbd839eba70a9ba61)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:35.969472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.021829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab38471172b4ccfbd839eba70a9ba61', 'request_id': 'req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:36.031451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab38471172b4ccfbd839eba70a9ba61', 'request_id': 'req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:36.039693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab38471172b4ccfbd839eba70a9ba61', 'request_id': 'req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:36.054043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] The token cache is being invalidate because project 6ab38471172b4ccfbd839eba70a9ba61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:36.054334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:36.055192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-69c1c3a9-d455-4ed7-87c2-fc6a5ee4db19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:36.056735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1108/2211] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:35 2022] DELETE /identity/v3/projects/6ab38471172b4ccfbd839eba70a9ba61 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:36.062092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261063-00b2-4122-978f-93ac56a52610 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.062408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261063-00b2-4122-978f-93ac56a52610 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.062712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261063-00b2-4122-978f-93ac56a52610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.082135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:36.089097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-86261063-00b2-4122-978f-93ac56a52610 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:36.091151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-86261063-00b2-4122-978f-93ac56a52610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:36.156770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1104/2212] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:36.164393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.164742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.165098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.186690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:36.187907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:36.197931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1109/2213] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:36.203087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261063-00b2-4122-978f-93ac56a52610 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.213262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261063-00b2-4122-978f-93ac56a52610 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.217857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.218447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.218727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.219026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.219675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:36.219977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.221550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.234709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1105/2214] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:36.240292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.251187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2dc0b-43c6-4e51-b07d-60f412c315e8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.257222 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.258050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.258444 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.258792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.259719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.262070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.298330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1110/2215] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:36.302411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.314174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac1cde-3202-4332-ae7d-bb861c82bb41 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.370838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.372160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.372443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.372713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.373326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.375543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.399198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1106/2216] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:36.400797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.411378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efaaa8-ce55-461c-95be-d9d7f0478125 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.416108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.416711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.416984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.417253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.417917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.420055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.433277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1111/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:36.435175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.445223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e0b85-3873-4fa8-af38-cb1c3d77512c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.449880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.450471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.450753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.451042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.451683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.453293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d20dcc-b69e-48d8-896a-384a10eae715 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.464230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1107/2218] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:36.469079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.479163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7383e-1902-4c43-a3af-5618f96bd62d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.484336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.484976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.485248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.485517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] PATH_INFO: `/v3/projects/af8005ca98fd4635bbf6b61b58a07305/users/cc41f0738962442280d934679b63b363/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.500778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8005ca98fd4635bbf6b61b58a07305, user_id=cc41f0738962442280d934679b63b363, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.503612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.521486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1112/2219] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:36 2022] PUT /identity/v3/projects/af8005ca98fd4635bbf6b61b58a07305/users/cc41f0738962442280d934679b63b363/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:36.533136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.533422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.533702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.562554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] MFA Rules not processed for user `cc41f0738962442280d934679b63b363`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:36.563795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:36.635532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1108/2220] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:36.640618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.693716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:36.710888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aadd8f-759c-441a-9d93-5b5f6cf8b3e8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.716106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.716703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.716976 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.717241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.717866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.719793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.756694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1113/2221] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:36.762478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.771422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:36.779472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb147a4a-ee0e-4371-b8f0-310bfdb24ee9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.850344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.850949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.851818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.852110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.852734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.855043 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.878487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1109/2222] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:36 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:36.882514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.892870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50ed29-1e85-454f-a1f0-77e56a571e79 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.897428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.898015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.898296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.898569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.899247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.901529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.912726 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1114/2223] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:36.917038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.929088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f8728-a6d4-49bb-a006-3b6405841228 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:36.934616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:36.935309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:36.935587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:36.935860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] PATH_INFO: `/v3/projects/466a3e9145ca49769e441233942c81c8/users/09c56f3f52ac45e4b4470b56563caba5/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:36.951948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=466a3e9145ca49769e441233942c81c8, user_id=09c56f3f52ac45e4b4470b56563caba5, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:36.955004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:36.975755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1110/2224] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:36 2022] PUT /identity/v3/projects/466a3e9145ca49769e441233942c81c8/users/09c56f3f52ac45e4b4470b56563caba5/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:36.976057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:36.986668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f044ec-c0a5-476d-98b0-cf3181f1f98a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.049113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.049724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.050012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.050319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.051489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.054165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ae953-22e4-407f-9ebd-e71ac992f88f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.071020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1115/2225] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:37.072845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.084930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57ef0-f5ce-41a7-b618-87ba56355c6b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.091144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.091923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.092113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.092383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] PATH_INFO: `/v3/projects/466a3e9145ca49769e441233942c81c8/users/09c56f3f52ac45e4b4470b56563caba5/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.094354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466a3e9145ca49769e441233942c81c8, user_id=09c56f3f52ac45e4b4470b56563caba5, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.097209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.109093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1111/2226] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:37 2022] PUT /identity/v3/projects/466a3e9145ca49769e441233942c81c8/users/09c56f3f52ac45e4b4470b56563caba5/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:37.120489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.120777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.122677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.158996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] MFA Rules not processed for user `09c56f3f52ac45e4b4470b56563caba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:37.160221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:37.255321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1116/2227] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:37.279326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.281105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:37.375838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf229-a434-4fdf-abf7-d2dd2d797f06 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.379729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.380394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.380757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.381085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.391801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.393387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c533-20c7-4526-abf3-9fda43bf81d7 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.404437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1112/2228] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:37.468542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:37.477941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35b475-801f-466b-ae1b-cb0388495119 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:37.482595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:37.483648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:37.484117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:37.484545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:37.535166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:37.553319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:37.554894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:37.564840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1117/2229] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:41.326699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:41.327337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:41.327521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:41.349616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:41.350561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:41.421773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1113/2230] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:58:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:41.478388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:41.480607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:41.567185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b8386-69f0-4269-9507-d0d80c713a00 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:41.572306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:41.572917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:41.573193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:41.573463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:41.584215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:41.585953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050da3c-8bb2-4af0-85ad-c280dfb6ce6c service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:41.595690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1118/2231] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:58:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:58:43.780328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:43.790793 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:43.909293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e78756-de4e-46af-8ffc-1c5badb2156c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:43.914833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:43.915958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:43.916481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:43.916933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] PATH_INFO: `/v3/users/cc41f0738962442280d934679b63b363` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:43.919112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc41f0738962442280d934679b63b363)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:43.932318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:44.017653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc41f0738962442280d934679b63b363'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.041719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc41f0738962442280d934679b63b363'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.042363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc41f0738962442280d934679b63b363'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.067834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc41f0738962442280d934679b63b363'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.086932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc41f0738962442280d934679b63b363'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.096559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1114/2232] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:43 2022] DELETE /identity/v3/users/cc41f0738962442280d934679b63b363 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:44.100997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:44.101466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:44.101555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:44.136538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:44.138562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:44.266429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1119/2233] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:44.277145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:44.279958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:44.357343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b893c-3b59-4cfb-a2c2-2b72dd623a0a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:44.361926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:44.362752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:44.363070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:44.363347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:44.373770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:44.376018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:44.395305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1115/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:58:44.706676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:44.709092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:44.728594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d321812-0d6d-4013-97d0-9d961bc15298 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:44.735310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:44.736368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:44.736800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:44.737518 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] PATH_INFO: `/v3/projects/af8005ca98fd4635bbf6b61b58a07305` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:44.738924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8005ca98fd4635bbf6b61b58a07305)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:44.741907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:44.850003 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8005ca98fd4635bbf6b61b58a07305', 'request_id': 'req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.864749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8005ca98fd4635bbf6b61b58a07305', 'request_id': 'req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.880076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8005ca98fd4635bbf6b61b58a07305', 'request_id': 'req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.885406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:44.901592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] The token cache is being invalidate because project af8005ca98fd4635bbf6b61b58a07305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:44.901592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.902870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:44.905257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1120/2235] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:44 2022] DELETE /identity/v3/projects/af8005ca98fd4635bbf6b61b58a07305 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:44.909112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:44.922268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc813ca-0e3b-4308-943f-8cc5f1acfc93 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:44.962239 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:45.047115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:45.047870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.048138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.051032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] PATH_INFO: `/v3/users/09c56f3f52ac45e4b4470b56563caba5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.052391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c56f3f52ac45e4b4470b56563caba5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:45.057813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:45.186267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c56f3f52ac45e4b4470b56563caba5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:45.203315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c56f3f52ac45e4b4470b56563caba5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:45.204740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c56f3f52ac45e4b4470b56563caba5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:45.242375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c56f3f52ac45e4b4470b56563caba5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:45.257232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f097cfa9-baff-42c6-815d-5a0e673f9275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c56f3f52ac45e4b4470b56563caba5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:45.272838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1116/2236] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:44 2022] DELETE /identity/v3/users/09c56f3f52ac45e4b4470b56563caba5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:45.741770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:45.746165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:45.799107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4de3-98e5-43ea-ac43-a64ee0c96d47 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:45.932984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:45.932984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:45.933325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:45.933422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] PATH_INFO: `/v3/projects/466a3e9145ca49769e441233942c81c8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:45.934890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466a3e9145ca49769e441233942c81c8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:45.936859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.015625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466a3e9145ca49769e441233942c81c8', 'request_id': 'req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:46.037318 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466a3e9145ca49769e441233942c81c8', 'request_id': 'req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:46.050514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466a3e9145ca49769e441233942c81c8', 'request_id': 'req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:46.071522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] The token cache is being invalidate because project 466a3e9145ca49769e441233942c81c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:46.071968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:46.072922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b47f0f2-1302-4a7b-aeb5-bfff18188c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:46.075363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1121/2237] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:45 2022] DELETE /identity/v3/projects/466a3e9145ca49769e441233942c81c8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:46.080395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.081041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.081246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.106532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:46.107811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:46.192359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1117/2238] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:46.201567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.202371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.202922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.233394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:46.235215 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:46.247670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1122/2239] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:46.252817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.267600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bbe4c-8e2f-4768-bb26-88fe816040c9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.277647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.278437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.278845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.279247 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.280066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:46.280497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.282366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.318629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1118/2240] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:46.323382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.336105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5427fad-ff36-4639-9fc2-7abb79c1e6e8 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.343575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.344197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.344615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.344890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.346197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.347482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.388914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1123/2241] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:46.392997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.407860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e34e5c-f218-456a-b467-51488c50ae23 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.468836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:46.516735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.517454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.517743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.518019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.519456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.521464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.561860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1119/2242] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:46.566179 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.584784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9187c36-88ca-41e1-9939-d70af6d4b05b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.590817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.591610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.592001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.592402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.593198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.594860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.609203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1124/2243] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:46.610996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.624775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8f78-3a6f-4a05-bcf9-66409989da48 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.638504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.639180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.640023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.640307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.641107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.642829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb234b-02b1-4fe3-9151-72b62f22fa94 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.655653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1120/2244] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:46.660501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.673854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8bf6e-2f5c-4efa-8216-1882c242555f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:46.679596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:46.680297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.680574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.680850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] PATH_INFO: `/v3/projects/539f1210d6fe430cb6b2fb780548234f/users/504e5edd202d48ed9387a094a1d0d953/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.701562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] RBAC: Authorizing `identity:create_grant(project_id=539f1210d6fe430cb6b2fb780548234f, user_id=504e5edd202d48ed9387a094a1d0d953, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:46.705034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:46.723822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1125/2245] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:46 2022] PUT /identity/v3/projects/539f1210d6fe430cb6b2fb780548234f/users/504e5edd202d48ed9387a094a1d0d953/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:46.742177 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:46.742792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:46.743788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:46.808141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] MFA Rules not processed for user `504e5edd202d48ed9387a094a1d0d953`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:46.810000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:46.894884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1121/2246] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:46.900066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:46.946132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:47.043422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22eec98-4505-482a-abbb-41c27b4e50de admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.052250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.053340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.053730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.054235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.055289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.061388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.100784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1126/2247] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:46 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:47.105822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.119575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb7150-ed3d-4748-bc2f-3f9d9d02d061 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.200266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.200892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.201199 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.201450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.202223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.204169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.230176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1122/2248] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:47 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:47.236869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.269453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221b0e-548b-4651-becf-5387f519445d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.281415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.284913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.285246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.285475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.287384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.292350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ee31c-ce60-4ddf-86ef-727fd78040ea admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.310197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1127/2249] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:47.312457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.313645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.314288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.314633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.328078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0d73c-fe78-4b3b-812a-856e741c03a2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.344417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.345332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.346174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.346831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] PATH_INFO: `/v3/projects/c473945ec04e4bd0954c19c5900fdf69/users/ddc446cd1aa14e47aaa7913f2557dc75/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.353420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] MFA Rules not processed for user `f7491479bd88425f8234e0b1d0e11e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:47.354651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:47.372746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c473945ec04e4bd0954c19c5900fdf69, user_id=ddc446cd1aa14e47aaa7913f2557dc75, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.376309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.394633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1123/2250] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:47 2022] PUT /identity/v3/projects/c473945ec04e4bd0954c19c5900fdf69/users/ddc446cd1aa14e47aaa7913f2557dc75/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:47.399517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.407251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:47.455248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1128/2251] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:47.560554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db38c5c-9f4d-418b-a0d4-7e07b1dc05cb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.565761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.566397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.566675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.566946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.567630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.569339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277a8fd-8758-4f0a-891c-8142ae351b42 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.580937 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1124/2252] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:47.586072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.608645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf862130-5455-4174-af5b-666efa93d9e3 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.615207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.616046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.616428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.617361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] PATH_INFO: `/v3/projects/c473945ec04e4bd0954c19c5900fdf69/users/ddc446cd1aa14e47aaa7913f2557dc75/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.619932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c473945ec04e4bd0954c19c5900fdf69, user_id=ddc446cd1aa14e47aaa7913f2557dc75, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.623221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:47.650472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.650834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.651273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.654202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1129/2253] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:47 2022] PUT /identity/v3/projects/c473945ec04e4bd0954c19c5900fdf69/users/ddc446cd1aa14e47aaa7913f2557dc75/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:47.724380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] MFA Rules not processed for user `ddc446cd1aa14e47aaa7913f2557dc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:47.726160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:47.848965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1125/2254] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:47.862725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:47.864843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:47.955204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc8613-2c16-41dd-b2c5-441c9c5f72bd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:47.958925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:47.959646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:47.960000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:47.960344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:47.982680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:47.988820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a93a3-8f87-4651-a0b3-92c2c02dfaaf service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:48.013946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1130/2255] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:49.086161 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:49.173649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22621e97-9a1e-4311-b4bf-ca80ae6073c4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:49.189042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:49.189777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:49.190034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:49.190303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] PATH_INFO: `/v3/users/504e5edd202d48ed9387a094a1d0d953` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:49.191542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=504e5edd202d48ed9387a094a1d0d953)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:49.194398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:49.238317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:49.270081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504e5edd202d48ed9387a094a1d0d953'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:49.280468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504e5edd202d48ed9387a094a1d0d953'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:49.281123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504e5edd202d48ed9387a094a1d0d953'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:49.296900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504e5edd202d48ed9387a094a1d0d953'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:49.305432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504e5edd202d48ed9387a094a1d0d953'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:49.313262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1126/2256] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:49 2022] DELETE /identity/v3/users/504e5edd202d48ed9387a094a1d0d953 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:49.317648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:49.318190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:49.318460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:49.340250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:49.341605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:49.371413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:49.421235 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1131/2257] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:49.431405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:49.433148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:49.516109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b182bae-843c-4df3-beba-8e967c4a54ec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:49.527737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:49.528416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:49.528699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:49.528971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:49.539156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:49.540631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:49.556765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1127/2258] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:50.055546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:50.058018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:50.093385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84aa92-1766-4b4a-82cf-ffbfa7a21239 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:50.101708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:50.103680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:50.104100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:50.104378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] PATH_INFO: `/v3/projects/539f1210d6fe430cb6b2fb780548234f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:50.105836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539f1210d6fe430cb6b2fb780548234f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:50.108967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:50.179192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539f1210d6fe430cb6b2fb780548234f', 'request_id': 'req-c6294bed-ebe2-4f06-999b-d8fe3123aca3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.188166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539f1210d6fe430cb6b2fb780548234f', 'request_id': 'req-c6294bed-ebe2-4f06-999b-d8fe3123aca3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.197447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539f1210d6fe430cb6b2fb780548234f', 'request_id': 'req-c6294bed-ebe2-4f06-999b-d8fe3123aca3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.217671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] The token cache is being invalidate because project 539f1210d6fe430cb6b2fb780548234f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:50.217671 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.218412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.220855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1132/2259] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:50 2022] DELETE /identity/v3/projects/539f1210d6fe430cb6b2fb780548234f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:50.224688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:50.235388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a904c2e-3d8c-494a-a08b-459c38de53c9 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:50.332633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:50.333545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:50.333902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:50.334326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] PATH_INFO: `/v3/users/ddc446cd1aa14e47aaa7913f2557dc75` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:50.335864 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc446cd1aa14e47aaa7913f2557dc75)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:50.338327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:50.427900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc446cd1aa14e47aaa7913f2557dc75'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.435309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:50.447643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc446cd1aa14e47aaa7913f2557dc75'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.450329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc446cd1aa14e47aaa7913f2557dc75'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.489452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc446cd1aa14e47aaa7913f2557dc75'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.517087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-031d4105-b51d-42a8-a1af-b4184084a67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc446cd1aa14e47aaa7913f2557dc75'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:50.534733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1128/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:58:50 2022] DELETE /identity/v3/users/ddc446cd1aa14e47aaa7913f2557dc75 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:51.050123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.052842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:51.078130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6294bed-ebe2-4f06-999b-d8fe3123aca3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.219546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.220303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.220584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.220908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] PATH_INFO: `/v3/projects/c473945ec04e4bd0954c19c5900fdf69` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.223105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c473945ec04e4bd0954c19c5900fdf69)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.225750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.325542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c473945ec04e4bd0954c19c5900fdf69', 'request_id': 'req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:51.333801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:51.336402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c473945ec04e4bd0954c19c5900fdf69', 'request_id': 'req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:51.346576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c473945ec04e4bd0954c19c5900fdf69', 'request_id': 'req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:51.380420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] The token cache is being invalidate because project c473945ec04e4bd0954c19c5900fdf69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:58:51.380844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:51.382337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-b20e1dd1-bd2d-4f65-8a66-60b33acee11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:58:51.386906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1133/2261] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:58:51 2022] DELETE /identity/v3/projects/c473945ec04e4bd0954c19c5900fdf69 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:51.391817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.392691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.393135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.426584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:51.427876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:51.607958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1129/2262] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:51.616249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.620104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.620104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.654714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:51.656775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:51.672292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1134/2263] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:51.676804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.723237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f701-3cd0-4696-8f46-54adf57f9fda None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.745121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.745841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.746262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.747408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.749353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:58:51.752994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.757732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.795496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1130/2264] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:58:51.802591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.817310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7f44f-337f-49ab-89a6-0a70306c081a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:51.823824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:51.824608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:51.825057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:51.825400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:51.826274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:51.829407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:51.893521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1135/2265] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:51 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:51.898130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:51.918142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047988e7-2313-43e6-950c-795d6b270151 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:52.074717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:52.076160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.076893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.077471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.078459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:52.084691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.125375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1131/2266] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:51 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:58:52.126002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:52.149918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38239b09-b6ec-4e2f-9947-dbb3ca6b2c93 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:52.159163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:52.160735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.161076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.161508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.163144 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:52.164816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.185100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1136/2267] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:52.185794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:52.202123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064e1f-4c7c-4f33-a3f1-304bfca76c9d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:52.209966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:52.211193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.211552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.211893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.212616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:52.214311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033fe204-3488-47a3-b374-9642f7ff8ed3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.235713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:52.258311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1132/2268] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:52.262597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087931a6-566d-4808-9234-0584f20a9407 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:52.281739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:52.282863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.283433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.283947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] PATH_INFO: `/v3/projects/ec4a769e7d6c4c19a0c631bd7e07cfc9/users/3db2595e05e348ff803c3ac8c7a5e89b/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.302294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4a769e7d6c4c19a0c631bd7e07cfc9, user_id=3db2595e05e348ff803c3ac8c7a5e89b, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:52.305299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.351544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1137/2269] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:52 2022] PUT /identity/v3/projects/ec4a769e7d6c4c19a0c631bd7e07cfc9/users/3db2595e05e348ff803c3ac8c7a5e89b/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:52.413123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.413854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.413854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.479632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] MFA Rules not processed for user `3db2595e05e348ff803c3ac8c7a5e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:52.481741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:52.539019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:52.678288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1133/2270] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:52.680014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:52.804688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbfe53-6448-4932-b815-f6968f03c7c1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:52.812722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:52.813586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:52.814157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:52.814560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:52.815365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:52.817814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:52.892321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:52.904682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1138/2271] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:58:52 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:52.911792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d92541-3027-4cd1-8d17-5ac102fe7938 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.051040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.051886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.052195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.052531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.053348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:53.055910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:53.109971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1134/2272] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:58:52 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:58:53.112126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:53.125114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd888b09-8139-4a23-986b-484de6509afa admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.141019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.143828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.144477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.145191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.146288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:53.149162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:53.172944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1139/2273] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:58:53.177339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:53.204084 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cca7eb-8a87-49dd-88c3-9a0d2ae7a5fb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.214232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.216085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.216493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.216865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] PATH_INFO: `/v3/projects/c6d5b9388fb040b382b1fb78413be7f2/users/9c7bcef53a2c4411ab9f83887a63a764/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.248835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d5b9388fb040b382b1fb78413be7f2, user_id=9c7bcef53a2c4411ab9f83887a63a764, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:53.252192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:53.282605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1135/2274] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:53 2022] PUT /identity/v3/projects/c6d5b9388fb040b382b1fb78413be7f2/users/9c7bcef53a2c4411ab9f83887a63a764/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:58:53.286484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:53.304440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef026e97-79bb-433d-a56c-627afabd592e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.373664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:58:53.404043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.404936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.405364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.405786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.406602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:53.408626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dfbd8-92a1-44d4-bb61-66b3374185f5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:53.422388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1140/2275] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:58:53.427424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:53.442628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05edae-5caa-414c-b6c8-cdd041a06679 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.459949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.461108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.461108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.461957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] PATH_INFO: `/v3/projects/c6d5b9388fb040b382b1fb78413be7f2/users/9c7bcef53a2c4411ab9f83887a63a764/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.467700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d5b9388fb040b382b1fb78413be7f2, user_id=9c7bcef53a2c4411ab9f83887a63a764, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:53.484794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e8a74-55fb-4ad3-97b4-11e7ed0eaf6d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:53.501800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1136/2276] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:58:53 2022] PUT /identity/v3/projects/c6d5b9388fb040b382b1fb78413be7f2/users/9c7bcef53a2c4411ab9f83887a63a764/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:58:53.511993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.513638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.514097 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.601888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.602723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.603393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:53.607167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] MFA Rules not processed for user `9c7bcef53a2c4411ab9f83887a63a764`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:53.609463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:53.642998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] MFA Rules not processed for user `478a0e3ac431440582d3dca1fe246ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:58:53.644364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:53.799369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1141/2277] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:58:53.801626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1137/2278] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:58:53.816238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:53.826669 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:53.957511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668cf6a-d622-4c4b-b81e-162672148e80 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:53.961467 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:53.962091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:53.962372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:53.962645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:54.109824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:54.111881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5401-089a-4938-9209-c8791a2984a6 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:54.127736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1142/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:58:59.348497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:58:59.350017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:58:59.364687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccea5a-48a4-4cb0-981d-37a15ccf4133 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:58:59.369003 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:58:59.369771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:58:59.370040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:58:59.370315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:58:59.382953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:58:59.384457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:58:59.411413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1138/2280] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:58:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:04.119621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:04.492601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:04.493408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:04.493854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:04.534146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:04.535624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:04.648481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1143/2281] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 17:59:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:04.665200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:04.667647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:04.689143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:04.712837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:04.839294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a79b7-8bc6-4adf-ba10-61d06cf33b82 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:04.855787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:04.856537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:04.856819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:04.857093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:04.873006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdae1b-570c-4cd8-adf0-99dc0d4a8fba None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:04.878644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:04.882599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f84dd-6f63-4885-9965-997f512f4dc6 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:04.895077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:04.895775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:04.896157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:04.896444 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] PATH_INFO: `/v3/users/3db2595e05e348ff803c3ac8c7a5e89b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:04.899775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db2595e05e348ff803c3ac8c7a5e89b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:04.904317 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:04.907999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1139/2282] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:59:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:59:05.051254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db2595e05e348ff803c3ac8c7a5e89b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:05.063980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db2595e05e348ff803c3ac8c7a5e89b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:05.065256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db2595e05e348ff803c3ac8c7a5e89b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:05.080454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db2595e05e348ff803c3ac8c7a5e89b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:05.116700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db2595e05e348ff803c3ac8c7a5e89b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:05.125901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1144/2283] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:59:04 2022] DELETE /identity/v3/users/3db2595e05e348ff803c3ac8c7a5e89b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:05.131428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:05.131709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:05.131991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:05.188312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:05.189037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:05.367769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1140/2284] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:05.380639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:05.382380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:05.430425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:05.477425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756d01b-52f3-4fa2-b0d8-185c1bc8f105 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:05.482137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:05.482848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:05.483185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:05.483453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:05.499020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:05.499508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:05.512948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1145/2285] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:06.003339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:06.004820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:06.021780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9d5d2-73fd-4a8a-8860-86b78977277f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:06.028780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:06.029568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:06.029915 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:06.030259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] PATH_INFO: `/v3/projects/ec4a769e7d6c4c19a0c631bd7e07cfc9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:06.031798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4a769e7d6c4c19a0c631bd7e07cfc9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:06.034452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:06.094506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4a769e7d6c4c19a0c631bd7e07cfc9', 'request_id': 'req-bf7e18a2-7d21-45a4-9626-c59a93a234fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.105176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4a769e7d6c4c19a0c631bd7e07cfc9', 'request_id': 'req-bf7e18a2-7d21-45a4-9626-c59a93a234fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.114879 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4a769e7d6c4c19a0c631bd7e07cfc9', 'request_id': 'req-bf7e18a2-7d21-45a4-9626-c59a93a234fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.127715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] The token cache is being invalidate because project ec4a769e7d6c4c19a0c631bd7e07cfc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:59:06.128004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.129195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.130721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1141/2286] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:59:06 2022] DELETE /identity/v3/projects/ec4a769e7d6c4c19a0c631bd7e07cfc9 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:06.134870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:06.145532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ac6c1-d050-4ff1-bbfe-31fb61e0f192 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:06.245941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:06.247424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:06.248415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:06.249429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] PATH_INFO: `/v3/users/9c7bcef53a2c4411ab9f83887a63a764` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:06.252690 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c7bcef53a2c4411ab9f83887a63a764)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:06.256349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:06.343400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c7bcef53a2c4411ab9f83887a63a764'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.351169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c7bcef53a2c4411ab9f83887a63a764'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.351874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c7bcef53a2c4411ab9f83887a63a764'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.361458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c7bcef53a2c4411ab9f83887a63a764'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.379481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d4309925-24db-4976-8dbc-b0b0476f880c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c7bcef53a2c4411ab9f83887a63a764'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:06.392170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1146/2287] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:59:06 2022] DELETE /identity/v3/users/9c7bcef53a2c4411ab9f83887a63a764 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:06.861829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:06.863799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:06.882760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e18a2-7d21-45a4-9626-c59a93a234fd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:06.902710 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:06.964238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:06.965549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:06.966057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:06.966554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] PATH_INFO: `/v3/projects/c6d5b9388fb040b382b1fb78413be7f2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:06.968581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d5b9388fb040b382b1fb78413be7f2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:06.972461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:07.068107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d5b9388fb040b382b1fb78413be7f2', 'request_id': 'req-4ee0149e-f772-4cc7-a0b1-94d3ac195756'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.083448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d5b9388fb040b382b1fb78413be7f2', 'request_id': 'req-4ee0149e-f772-4cc7-a0b1-94d3ac195756'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.094747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d5b9388fb040b382b1fb78413be7f2', 'request_id': 'req-4ee0149e-f772-4cc7-a0b1-94d3ac195756'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.122485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] The token cache is being invalidate because project c6d5b9388fb040b382b1fb78413be7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:59:07.123005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.124674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.127605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1142/2288] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:59:06 2022] DELETE /identity/v3/projects/c6d5b9388fb040b382b1fb78413be7f2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:07.536050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.537762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.538301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.575766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:07.578297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:07.609754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.611174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:07.759684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:07.786220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0149e-f772-4cc7-a0b1-94d3ac195756 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:07.790325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1147/2289] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:59:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:07.794447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:07.795376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.795812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.796411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] PATH_INFO: `/v3/users/ff0c1d9939374d14bbc7cacad41162ea` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.797843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0c1d9939374d14bbc7cacad41162ea)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:07.800256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:07.806592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:07.808143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:07.881176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0c1d9939374d14bbc7cacad41162ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.886251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:07.893001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0c1d9939374d14bbc7cacad41162ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.895615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0c1d9939374d14bbc7cacad41162ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.913273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03c7d0-6b9c-4476-bb79-9680427cbcb6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:07.917153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0c1d9939374d14bbc7cacad41162ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.921929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:07.923226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:07.923612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:07.923986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:07.929906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:07.931282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0c1d9939374d14bbc7cacad41162ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:07.940987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1143/2290] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:59:07 2022] DELETE /identity/v3/users/ff0c1d9939374d14bbc7cacad41162ea => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:59:07.948672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:07.951403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61e7dc-3591-48c0-9303-d97a7df9258a service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:07.961773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1148/2291] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:59:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 17:59:08.224057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.226512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:08.334022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2c64e-9f83-4896-a90c-9c3e1d9ace3e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.340611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.341557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.342089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.342378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] PATH_INFO: `/v3/projects/c00730b67ae04f58be1322a80cc6a4c2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.343596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00730b67ae04f58be1322a80cc6a4c2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.345560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.419909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00730b67ae04f58be1322a80cc6a4c2', 'request_id': 'req-7a214c71-3345-4955-aea2-2a9dbbb2aa38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:08.451024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00730b67ae04f58be1322a80cc6a4c2', 'request_id': 'req-7a214c71-3345-4955-aea2-2a9dbbb2aa38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:08.466074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00730b67ae04f58be1322a80cc6a4c2', 'request_id': 'req-7a214c71-3345-4955-aea2-2a9dbbb2aa38'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:08.496101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] The token cache is being invalidate because project c00730b67ae04f58be1322a80cc6a4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:59:08.496443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:08.498261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a214c71-3345-4955-aea2-2a9dbbb2aa38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:08.504490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1144/2292] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:59:08 2022] DELETE /identity/v3/projects/c00730b67ae04f58be1322a80cc6a4c2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:08.507800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.508421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.509015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.568276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:08.569718 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:08.671385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1149/2293] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:08.672610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.673122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.673446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.731014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:08.731014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:08.748068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1145/2294] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:08.752833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.774411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c080da-a101-4e82-8d37-275147c37f9a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.782799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.784240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.784868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.785380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.786261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:59:08.787048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.788903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.818233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.828021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1150/2295] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 17:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:59:08.832688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e7b88-76ae-4f2e-9fcf-0d8f2cd45309 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:08.840932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:08.841972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:08.842481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:08.843028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:08.844095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:08.848625 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:08.905712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1146/2296] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 17:59:08 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:59:08.910232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:08.927648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344204c-cbbf-4ddd-9389-0e73630d6058 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:09.000596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:09.001296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.001629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.001971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.002785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:09.004824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:09.023364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:09.040956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1151/2297] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 17:59:08 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:59:09.042669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:09.061920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61e8cf-d01b-4e34-988e-6eb005c5b8d8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:09.081482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:09.082038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.087853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.089055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.089055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:09.090746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:09.103593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1147/2298] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:59:09.108401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:09.128279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf000de1-14b6-443e-8ae0-53e4e50ca783 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:09.134397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:09.135198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.135604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.136262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.137447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:09.140796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48925-f639-4eee-b23c-f82176c6af75 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:09.163026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1152/2299] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 17:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:59:09.165892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:09.178854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33305a-b073-4fba-a851-d0dbdd144563 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:09.185655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:09.186568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.186903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.187145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] PATH_INFO: `/v3/projects/d95d6b2fb1654424bf048368c115fc5f/users/f9e414c2f6de4c67a3dfd75a7d8ffec9/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.219330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95d6b2fb1654424bf048368c115fc5f, user_id=f9e414c2f6de4c67a3dfd75a7d8ffec9, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:09.219330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:09.220735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:09.232368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1148/2300] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 17:59:09 2022] PUT /identity/v3/projects/d95d6b2fb1654424bf048368c115fc5f/users/f9e414c2f6de4c67a3dfd75a7d8ffec9/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:09.308393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.309115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.309474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.345588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] MFA Rules not processed for user `f9e414c2f6de4c67a3dfd75a7d8ffec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:09.346811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:09.437854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1153/2301] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:59:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:59:09.458771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:09.460738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:09.533865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181e46d-37fe-44f2-8f50-7b234cb63812 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:09.537742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:09.538432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:09.538802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:09.539221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:09.550339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:09.551857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:09.562693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1149/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:11.713605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:11.723339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:11.912698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82843906-7526-4447-8b0d-661ef65ff7ac None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:11.924792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:11.928865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:11.929468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:11.930333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:11.935321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:11.955387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:11.958059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1677f-274b-43c6-a669-fddfe4e03711 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:11.977756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1154/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:12.469428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:12.618951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046d2ea-e8b4-42fd-a6f8-82bfd7a855ec service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:12.625166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:12.625824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:12.626096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:12.626372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] PATH_INFO: `/v3/users/f9e414c2f6de4c67a3dfd75a7d8ffec9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:12.627549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e414c2f6de4c67a3dfd75a7d8ffec9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:12.629561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:12.712972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e414c2f6de4c67a3dfd75a7d8ffec9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:12.727156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e414c2f6de4c67a3dfd75a7d8ffec9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:12.728465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e414c2f6de4c67a3dfd75a7d8ffec9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:12.750022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e414c2f6de4c67a3dfd75a7d8ffec9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:12.759891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e414c2f6de4c67a3dfd75a7d8ffec9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:12.770807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1150/2304] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 17:59:12 2022] DELETE /identity/v3/users/f9e414c2f6de4c67a3dfd75a7d8ffec9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:59:12.773310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:12.773780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:12.773898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:12.801543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:12.803529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:12.899833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1155/2305] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 17:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:12.911323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:12.916384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:12.973729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:13.049995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17656788-0240-4f3b-9e0f-e3a1229e38fd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:13.054670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:13.055384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.055670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.055982 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.088679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:13.090315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7acd-1129-4d9c-b756-20cd7cf3b4cd service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:13.106815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1151/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:13.914523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:13.916796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:13.944653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fdf91-e52f-4edd-ae0a-f5d5765e7aed None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:13.952844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:13.953502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:13.953781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:13.954054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] PATH_INFO: `/v3/projects/d95d6b2fb1654424bf048368c115fc5f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:13.955377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95d6b2fb1654424bf048368c115fc5f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:13.957484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:13.959221 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:14.106678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95d6b2fb1654424bf048368c115fc5f', 'request_id': 'req-bdeb0f64-332b-494e-a7f5-d925547afe96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:14.120136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95d6b2fb1654424bf048368c115fc5f', 'request_id': 'req-bdeb0f64-332b-494e-a7f5-d925547afe96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:14.133410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95d6b2fb1654424bf048368c115fc5f', 'request_id': 'req-bdeb0f64-332b-494e-a7f5-d925547afe96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:14.155426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] The token cache is being invalidate because project d95d6b2fb1654424bf048368c115fc5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:59:14.155919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:14.156626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bdeb0f64-332b-494e-a7f5-d925547afe96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:59:14.158586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1156/2307] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 17:59:13 2022] DELETE /identity/v3/projects/d95d6b2fb1654424bf048368c115fc5f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:59:23.098002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=18 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:23.415905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.416183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.416460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.417788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1152/2308] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:23.424072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5ab6-9423-4796-8a86-01882b426003 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.424587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5ab6-9423-4796-8a86-01882b426003 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.424684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5ab6-9423-4796-8a86-01882b426003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.447054 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5a6c5ab6-9423-4796-8a86-01882b426003 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:23.448256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5a6c5ab6-9423-4796-8a86-01882b426003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:23.523340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1157/2309] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:59:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:23.535330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:23.537190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:23.620783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f586ff-e3f3-4a07-be32-c39df470d47c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:23.625433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:23.626047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.626328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.626605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.637973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:23.639630 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb84845-c385-42e5-9a74-e08704259e09 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:23.650398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1153/2310] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:23.723133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.723446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.723727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.725085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1158/2311] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:23.730119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d664-0d6a-4270-812b-efe63b9954e0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.730489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d664-0d6a-4270-812b-efe63b9954e0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.730842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d664-0d6a-4270-812b-efe63b9954e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.753159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b349d664-0d6a-4270-812b-efe63b9954e0 None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:23.754406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b349d664-0d6a-4270-812b-efe63b9954e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:23.809025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1154/2312] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:59:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:23.821491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:23.823307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:23.837741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963888ab-2bcd-4884-a64c-8dd943563dbe None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:23.841213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:23.841799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.842069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.842334 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.849614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:23.851101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e4d18-8ad3-456f-ac22-89ab7b1d9c60 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:23.859502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1159/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:23.943225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f198d6-8560-4728-b63a-ab06935996d4 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.943518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f198d6-8560-4728-b63a-ab06935996d4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.943792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f198d6-8560-4728-b63a-ab06935996d4 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.945166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1155/2314] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 17:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:59:23.950740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2889b61-84f2-49d1-aaeb-5f4cec072a54 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:23.951081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2889b61-84f2-49d1-aaeb-5f4cec072a54 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:23.951362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2889b61-84f2-49d1-aaeb-5f4cec072a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:23.970521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f2889b61-84f2-49d1-aaeb-5f4cec072a54 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:23.971748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2889b61-84f2-49d1-aaeb-5f4cec072a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:23.980919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1160/2315] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 17:59:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:23.992021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f198d6-8560-4728-b63a-ab06935996d4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:24.003800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f198d6-8560-4728-b63a-ab06935996d4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:24.007479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:24.008077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:24.008361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:24.008663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:24.019210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:24.020629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:24.031832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1156/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 17:59:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:59:26.845704 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947bf0-f825-4dfc-b2eb-3e5fd4570857 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:26.846190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947bf0-f825-4dfc-b2eb-3e5fd4570857 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:26.846295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947bf0-f825-4dfc-b2eb-3e5fd4570857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:26.865719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f7947bf0-f825-4dfc-b2eb-3e5fd4570857 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:26.866914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f7947bf0-f825-4dfc-b2eb-3e5fd4570857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:26.921683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1161/2317] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 17:59:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:26.933493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:26.934991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:27.014072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9b03e-2415-47dd-9145-a96da377de66 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:27.019174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:27.019778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:27.020060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:27.020333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:27.030041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:27.031681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:27.040262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1157/2318] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:59:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:59:36.914569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:37.036853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:59:58.069102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c723-8727-40ef-a06f-08ab2c0d7d4b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:58.069610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c723-8727-40ef-a06f-08ab2c0d7d4b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:58.069699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c723-8727-40ef-a06f-08ab2c0d7d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:58.089705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0354c723-8727-40ef-a06f-08ab2c0d7d4b None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:59:58.091672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0354c723-8727-40ef-a06f-08ab2c0d7d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:59:58.098576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1162/2319] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 17:59:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:59:58.120278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:59:58.131371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509903-07e0-4830-bbf3-00ff47d12e7a service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:59:58.136540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:59:58.137126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:59:58.137397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:59:58.137659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:59:58.146137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:59:58.147725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad6c8f-490a-42ff-8dd9-bb14e7d46f60 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:59:58.155464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1158/2320] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 17:59:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:00:08.084338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:08.152430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:27.513237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3602d-4449-4deb-b725-2dadd34aa4cf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:27.513807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3602d-4449-4deb-b725-2dadd34aa4cf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:27.513807 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3602d-4449-4deb-b725-2dadd34aa4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:27.533189 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f9f3602d-4449-4deb-b725-2dadd34aa4cf None None] MFA Rules not processed for user `478a0e3ac431440582d3dca1fe246ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:27.534390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f9f3602d-4449-4deb-b725-2dadd34aa4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:27.598119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1163/2321] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:00:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:00:37.591847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=25 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:38.545958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:38.546484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:38.546484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:38.568473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] MFA Rules not processed for user `f7491479bd88425f8234e0b1d0e11e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:38.569651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:38.624507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1159/2322] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:00:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:00:48.617661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=25 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:00:56.088028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:00:56.088530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:00:56.088971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:00:56.110990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] MFA Rules not processed for user `18bf10b172e7450a9b618683643ed48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:00:56.112247 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:00:56.172312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1164/2323] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:01:03.984512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:03.994659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d065d4-4cf8-48c9-94f1-4ccd15947469 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:03.998597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:03.999213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:03.999488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:03.999758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:04.001791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:04.015838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:04.017255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:04.028739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1160/2324] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:06.164919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=25 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:06.418470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:06.427932 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452579c6-c2a3-4ac5-a991-067ab0cbe5be None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:06.431413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:06.431998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:06.432272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:06.432535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:06.434216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:06.447793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:06.449204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:06.460856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1165/2325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:14.023859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:16.455249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:25.961689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:25.963150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:26.018134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c018b7e-64a7-480a-b5ee-a93c8766aa60 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:26.021786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:26.022887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:26.023188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:26.023456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:26.025128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:26.035701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:26.037104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:26.046755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1161/2326] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:27.972028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:27.982031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8fcc3-2def-4fb4-b314-a353efa70467 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:27.985677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:27.986271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:27.986548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:27.986817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:27.988681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:28.048878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:28.050305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2952e6-c6a9-4c33-b6c0-844925463796 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:28.060448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1166/2327] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:28.314895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:28.316406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:28.334048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0c205-18f4-485e-bafa-6fd6563937f0 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:28.344317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:28.346106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:28.346106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:28.346106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:28.348062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:28.359632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:28.361208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:28.372136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1162/2328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:38.055917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=28 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:38.365994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:01:42.080083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:42.080597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:42.080692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:42.099266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] MFA Rules not processed for user `cb11c04acb4f40cd9d09edf16b11abb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:01:42.100461 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:42.107449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1167/2329] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:01:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:01:42.433916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:42.443318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dc513-1e9e-440c-93a3-d596c38011e3 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:42.447967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:42.448701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:42.448987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:42.449259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:42.451088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:42.461771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:42.463372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:42.472283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1163/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:43.039911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:43.041260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:43.113834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc94f1-e91d-449b-82cb-68cc154b3e80 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:43.118805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:43.119433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:43.119712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:43.120156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:43.121966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:43.134526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:43.136137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:43.145954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1168/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:46.297225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:46.319663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c8c6-394f-43f0-bd34-83ff6aa19e32 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:46.336626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:46.337353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:46.337705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:46.338051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:46.343695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:46.371730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:46.373409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:46.391162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1164/2332] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:47.081579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:47.092395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668c2c-a39f-47ae-8520-32f604da49c0 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:47.096194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:47.096800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:47.097074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:47.097397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:47.099168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:47.111035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:47.112504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:47.122040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1169/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:47.926405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:47.927905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:47.941966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa3d5e-edfe-42dd-929b-f8adfb1f2e21 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:47.948222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:47.948829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:47.949108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:47.949382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:47.951136 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:47.963620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:47.965185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:47.975381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1165/2334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:55.540041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:55.542004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:55.554327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277bf8-e77f-424c-b22e-4a98ed4a15a3 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:55.559035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:55.559963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:55.560435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:55.560785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:55.563217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:55.643468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:55.645068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:55.658042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1170/2335] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:56.259193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:56.270030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074c1b8-f42b-4bf8-9c63-fdef06a2493a service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:56.275977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:56.276596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:56.276872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:56.277141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:56.279158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:56.295665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:56.297454 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:56.312258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1166/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:57.345760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:57.362094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e56ce-bc3a-4f40-bb23-1dbe3255b3b6 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:57.373134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:57.374536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:57.374944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:57.376145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:57.379431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:57.395395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:57.397423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf7cd94-02fe-485c-bbba-d2b37c8d241b service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:57.425391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1171/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:01:59.805996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:01:59.807419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:01:59.823468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacc6cd-ff4f-499a-912f-81ab99b32072 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:01:59.830646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:01:59.831429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:01:59.831803 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:01:59.832144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] PATH_INFO: `/v3/users/f7491479bd88425f8234e0b1d0e11e55` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:01:59.833347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7491479bd88425f8234e0b1d0e11e55)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:01:59.835310 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:01:59.891503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7491479bd88425f8234e0b1d0e11e55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:01:59.899752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7491479bd88425f8234e0b1d0e11e55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:01:59.900426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7491479bd88425f8234e0b1d0e11e55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:01:59.914384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7491479bd88425f8234e0b1d0e11e55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:01:59.921250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7491479bd88425f8234e0b1d0e11e55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:01:59.927093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1167/2338] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:01:59 2022] DELETE /identity/v3/users/f7491479bd88425f8234e0b1d0e11e55 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:00.009026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a331-94ac-4206-9df0-407d62f16603 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.009595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a331-94ac-4206-9df0-407d62f16603 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.009595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8a331-94ac-4206-9df0-407d62f16603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.028497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e3a8a331-94ac-4206-9df0-407d62f16603 None None] MFA Rules not processed for user `478a0e3ac431440582d3dca1fe246ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:00.029682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e3a8a331-94ac-4206-9df0-407d62f16603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:00.085019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:00.097169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1172/2339] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:00.210521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.212775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:00.290326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db82e7-f5b4-4820-b2f3-f8c7023ac0f9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.295121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.295751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.296049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.296321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] PATH_INFO: `/v3/projects/13cf0de2979c44a08f500430834617d2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.297433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cf0de2979c44a08f500430834617d2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.299405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.324275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:00.360297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cf0de2979c44a08f500430834617d2', 'request_id': 'req-754d673a-87cc-4fcb-9924-8f3d86c25013'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:00.369230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cf0de2979c44a08f500430834617d2', 'request_id': 'req-754d673a-87cc-4fcb-9924-8f3d86c25013'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:00.377056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cf0de2979c44a08f500430834617d2', 'request_id': 'req-754d673a-87cc-4fcb-9924-8f3d86c25013'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:00.392868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] The token cache is being invalidate because project 13cf0de2979c44a08f500430834617d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:02:00.393322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:00.395397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-754d673a-87cc-4fcb-9924-8f3d86c25013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:00.397228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1168/2340] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:02:00 2022] DELETE /identity/v3/projects/13cf0de2979c44a08f500430834617d2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:00.402229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.402709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.403164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.425713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:00.426921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:00.493399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1173/2341] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:02:00.501385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.501673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.501953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.522890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:00.524179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:00.532732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1169/2342] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:00.537004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.547920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba7a2e-7ffc-41b1-b113-097ef1f13d47 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.552740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.553410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.553778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.554209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.555538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:02:00.555904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.557738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.570818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1174/2343] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:02:00.575025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.584731 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01159f4-376f-4386-b8e5-6f270c8fe632 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.589657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.590265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.590540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.590807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.591487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.593380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.628953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1170/2344] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:00.630115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.642594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1f5c-7764-4da8-8d97-75725bd5e162 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.713018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.713649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.714129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.714243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.714845 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.717001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.743804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1175/2345] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:00.748266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.763093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de517e01-8901-4148-8980-96ef8fc8061e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.769839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.770686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.771110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.771450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.772250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.774421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.788984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1171/2346] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:00.793324 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.807920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0d04b-ab8d-4c28-b021-a204a94b7509 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.814167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.814770 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.815270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.815750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.816764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.818741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9cbeb-0e5d-4c3a-a6c8-c909cdf6af37 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.838232 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:00.850705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafa026-efc8-4b78-949c-4bb995dddc0d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:00.851001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1176/2347] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:00.870094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:00.870872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.871840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.872263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] PATH_INFO: `/v3/projects/987570eb14f64ebb8719acd473a3ffb4/users/32a793fa6bb1458c9d393a81bcde5ab2/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.894211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987570eb14f64ebb8719acd473a3ffb4, user_id=32a793fa6bb1458c9d393a81bcde5ab2, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:00.897814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:00.918457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1172/2348] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:00 2022] PUT /identity/v3/projects/987570eb14f64ebb8719acd473a3ffb4/users/32a793fa6bb1458c9d393a81bcde5ab2/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:00.928088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:00.928643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:00.929140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:00.959877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] MFA Rules not processed for user `32a793fa6bb1458c9d393a81bcde5ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:00.962293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:01.032648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:01.055747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1177/2349] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:01.062553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.154900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1202a0-23b2-4ced-93e1-f2b73e01cbc2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.161482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.162218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.162769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.163552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.164642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.167432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.198655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1173/2350] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:02:01 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:01.202758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.214986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7aeb9-7f03-4342-8ee6-70a0ac7bdca2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.286176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.286879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.287242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.287575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.288259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.290330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.316903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1178/2351] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:02:01 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:01.318460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.329641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34ecd6-3c1f-4b89-89d8-6a2086f5bd6d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.334504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.335125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.335858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.335858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.336477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.339717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.350476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1174/2352] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:01.354466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.366560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e2265-827e-4e8e-9afc-19b381f53721 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.375018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.376088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.376446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.376787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] PATH_INFO: `/v3/projects/4b7efae187914a289fc2b8c0aec6f680/users/1e06fa9bae9e401587b6def73b2c29d4/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.395499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7efae187914a289fc2b8c0aec6f680, user_id=1e06fa9bae9e401587b6def73b2c29d4, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.398391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.415637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1179/2353] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:01 2022] PUT /identity/v3/projects/4b7efae187914a289fc2b8c0aec6f680/users/1e06fa9bae9e401587b6def73b2c29d4/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:01.418600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.429206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dd541-f2fc-48dd-aafe-c05690cdf9df admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.451197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:01.505515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.506169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.506445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.506709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.507403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.510154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd81d2-c24a-4c6c-9fc2-477716cb02e1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.521697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1175/2354] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:01.526478 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.540950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bcddf-4bff-4888-9680-5c171a474212 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.545732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.546479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.546832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.547199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] PATH_INFO: `/v3/projects/4b7efae187914a289fc2b8c0aec6f680/users/1e06fa9bae9e401587b6def73b2c29d4/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.549273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7efae187914a289fc2b8c0aec6f680, user_id=1e06fa9bae9e401587b6def73b2c29d4, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.552096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.562763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1180/2355] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:01 2022] PUT /identity/v3/projects/4b7efae187914a289fc2b8c0aec6f680/users/1e06fa9bae9e401587b6def73b2c29d4/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:01.574604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.574897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.575207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.604157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] MFA Rules not processed for user `1e06fa9bae9e401587b6def73b2c29d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:01.605302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:01.691017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1176/2356] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:01.702177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.704604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:01.763914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9939-4364-4bd4-a413-82c00acc98f6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.768420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.769028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.769307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.769580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:01.780301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:01.781758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c8fb8-fd67-40aa-8417-bdecdc44334d service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:01.792156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1181/2357] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:01.907027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:01.923899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf47f7-d8cb-4381-8c2a-f83eaea01074 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:01.934917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:01.937523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:01.937846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:01.938306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:02.063657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:02.071502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:02.083501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1177/2358] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:11.792608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:12.078261 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=82 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:22.758890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402cce2-950f-43da-bb8e-2be47ddb8b4b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:22.759387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402cce2-950f-43da-bb8e-2be47ddb8b4b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:22.759604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402cce2-950f-43da-bb8e-2be47ddb8b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:22.782083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9402cce2-950f-43da-bb8e-2be47ddb8b4b None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:22.783332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9402cce2-950f-43da-bb8e-2be47ddb8b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:22.842888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1182/2359] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:02:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:22.892466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:22.893906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:22.981111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f2d2-e1e5-4ae2-8d27-dadf590ef22b service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:22.986429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:22.987112 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:22.987393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:22.987669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:22.996251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:22.997831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edfcd4-be0f-4554-812b-8a7f2b531015 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:23.006445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1178/2360] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:02:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:02:32.836534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=25 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:33.003053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:36.687567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:36.688142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:36.688251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:36.689855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1183/2361] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 18:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:02:36.697196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286471-375a-4376-9860-fe43c4dc6ea6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:36.697641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286471-375a-4376-9860-fe43c4dc6ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:36.697739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286471-375a-4376-9860-fe43c4dc6ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:36.717897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b3286471-375a-4376-9860-fe43c4dc6ea6 None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:36.719096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b3286471-375a-4376-9860-fe43c4dc6ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:36.802677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1179/2362] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:36.816374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:36.817911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:36.880922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6c22d-420b-42c2-afcf-08907e941dd5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:36.886530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:36.887283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:36.887568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:36.888050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:36.900335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:36.902243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfe9e7-cf24-4973-ae0e-81d19ce676c1 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:36.912780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1184/2363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:36.991158 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:36.991360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:36.991709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:36.993203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1180/2364] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 18:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:02:36.998116 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54012c-a929-4079-9519-63c0c764a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:36.998405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54012c-a929-4079-9519-63c0c764a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:36.998680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54012c-a929-4079-9519-63c0c764a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:37.018771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0d54012c-a929-4079-9519-63c0c764a8b0 None None] MFA Rules not processed for user `768dabebad264236b8f43dbd1c85a964`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:37.019997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d54012c-a929-4079-9519-63c0c764a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:37.075329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1185/2365] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 18:02:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:37.087886 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:37.089619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:37.103346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d465a8b-5d9b-410c-a6dd-99f76a38f22e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:37.106913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:37.107551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:37.107832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:37.108105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:37.116159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:37.117620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cb39f-8112-4ea7-9c8a-0422ee8aa136 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:37.126723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1181/2366] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:37.217900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b5d7d-b067-42e2-9a6b-7bbd589fcb9c None None] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:37.218188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b5d7d-b067-42e2-9a6b-7bbd589fcb9c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:37.218466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b5d7d-b067-42e2-9a6b-7bbd589fcb9c None None] PATH_INFO: `/` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:37.219855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1186/2367] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 9 18:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 18:02:37.225254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a986-b5f8-4800-866d-2057cc614d6b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:37.225535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a986-b5f8-4800-866d-2057cc614d6b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:37.225816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860a986-b5f8-4800-866d-2057cc614d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:37.245857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2860a986-b5f8-4800-866d-2057cc614d6b None None] MFA Rules not processed for user `a136641c41464f1f969fe6c16dfe3501`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:37.246990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2860a986-b5f8-4800-866d-2057cc614d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:37.256457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1182/2368] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 9 18:02:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:37.267412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b5d7d-b067-42e2-9a6b-7bbd589fcb9c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:37.278256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b5d7d-b067-42e2-9a6b-7bbd589fcb9c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:37.281818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:37.282399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:37.282676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:37.282946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:37.293196 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:37.294664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:37.305524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1187/2369] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:41.237347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:41.237870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:41.237870 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:41.257009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:41.258326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:41.266058 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1183/2370] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:02:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:41.278217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:41.289294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c9ae3-6b8d-4a98-99d7-a571cd802979 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:41.294185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:41.294767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:41.295075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:41.295352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:41.303190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:41.304760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c2da6-963e-48f1-95e0-521339ea6527 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:41.312842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1188/2371] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:02:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:02:43.396491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:43.489669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a9a1f-065d-43c0-92d5-435d62991a4b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:43.495339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:43.495973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:43.496254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:43.496553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] PATH_INFO: `/v3/users/32a793fa6bb1458c9d393a81bcde5ab2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:43.497705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a793fa6bb1458c9d393a81bcde5ab2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:43.499676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:43.564901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a793fa6bb1458c9d393a81bcde5ab2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:43.572236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a793fa6bb1458c9d393a81bcde5ab2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:43.572827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a793fa6bb1458c9d393a81bcde5ab2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:43.584656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:43.587067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a793fa6bb1458c9d393a81bcde5ab2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:43.596322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a793fa6bb1458c9d393a81bcde5ab2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:43.606518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1184/2372] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:02:43 2022] DELETE /identity/v3/users/32a793fa6bb1458c9d393a81bcde5ab2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:43.608362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:43.608861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:43.609033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:43.630192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:43.631589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:43.694825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1189/2373] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:43.705005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:43.712202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:43.788699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a125b-4325-4aac-ae0c-3fc4794ecaa7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:43.793039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:43.793649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:43.793927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:43.794196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:43.804097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:43.805546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:43.816746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1185/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:44.170291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:44.172613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:44.199204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf35632-06ca-40b1-b95b-3d4a8ffca913 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:44.206660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:44.207434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:44.207855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:44.208254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] PATH_INFO: `/v3/projects/987570eb14f64ebb8719acd473a3ffb4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:44.210652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=987570eb14f64ebb8719acd473a3ffb4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:44.213831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:44.243426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:44.294202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987570eb14f64ebb8719acd473a3ffb4', 'request_id': 'req-cad10a03-10cf-49b2-a535-ab4258bb1cee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.300481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987570eb14f64ebb8719acd473a3ffb4', 'request_id': 'req-cad10a03-10cf-49b2-a535-ab4258bb1cee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.312035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987570eb14f64ebb8719acd473a3ffb4', 'request_id': 'req-cad10a03-10cf-49b2-a535-ab4258bb1cee'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.345178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] The token cache is being invalidate because project 987570eb14f64ebb8719acd473a3ffb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:02:44.345483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.346143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.350227 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1190/2375] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:02:44 2022] DELETE /identity/v3/projects/987570eb14f64ebb8719acd473a3ffb4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:44.354287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:44.364408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff6bb6-643f-496c-8e63-2f6eef8c95d7 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:44.449858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:44.450562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:44.450830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:44.451141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] PATH_INFO: `/v3/users/1e06fa9bae9e401587b6def73b2c29d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:44.452307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e06fa9bae9e401587b6def73b2c29d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:44.454763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:44.510223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e06fa9bae9e401587b6def73b2c29d4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.518656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e06fa9bae9e401587b6def73b2c29d4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.519377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e06fa9bae9e401587b6def73b2c29d4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.537487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e06fa9bae9e401587b6def73b2c29d4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.544788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c6672e7b-96e1-4644-856a-1f4ef209ee0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e06fa9bae9e401587b6def73b2c29d4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:44.551887 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1186/2376] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:02:44 2022] DELETE /identity/v3/users/1e06fa9bae9e401587b6def73b2c29d4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:44.850740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:44.852886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:44.866352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad10a03-10cf-49b2-a535-ab4258bb1cee admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:44.941656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:44.942203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:44.942500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:44.942768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] PATH_INFO: `/v3/projects/4b7efae187914a289fc2b8c0aec6f680` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:44.944065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7efae187914a289fc2b8c0aec6f680)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:44.945960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.001293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7efae187914a289fc2b8c0aec6f680', 'request_id': 'req-f94af6f5-c5bd-465f-90e7-2d15a444c878'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:45.006649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7efae187914a289fc2b8c0aec6f680', 'request_id': 'req-f94af6f5-c5bd-465f-90e7-2d15a444c878'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:45.014421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7efae187914a289fc2b8c0aec6f680', 'request_id': 'req-f94af6f5-c5bd-465f-90e7-2d15a444c878'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:45.035223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] The token cache is being invalidate because project 4b7efae187914a289fc2b8c0aec6f680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:02:45.035508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:45.036132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f94af6f5-c5bd-465f-90e7-2d15a444c878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:02:45.037597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1191/2377] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:02:44 2022] DELETE /identity/v3/projects/4b7efae187914a289fc2b8c0aec6f680 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:45.043024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.043339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.043612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.058304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:45.066127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:45.071256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:45.143961 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1187/2378] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:45.153213 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.153498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.153793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.174779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:45.176004 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:45.186600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1192/2379] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:45.192251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.202632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf53b6-e1e1-4241-b435-b784566919d0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.207463 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.208081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.208358 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.208631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.209373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:02:45.209686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.211421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.224241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1188/2380] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:02:45.229699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.239204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13f0d-6c95-4236-b5ce-07d76ffd4322 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.244410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.245012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.245283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.245575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.246200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.248135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.281781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1193/2381] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:45.285878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.297219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c69f4-750f-49ed-bd98-bb61554b0ed7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.361567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.362257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.362536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.362810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.363477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.365435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.391810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1189/2382] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/users => generated 278 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:02:45.393007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.410483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dcf1f-777b-49a4-aead-88afacc44c20 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.416474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.417575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.418074 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.418627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.419342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.420940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.427882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:45.435123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1194/2383] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:02:45.437904 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.448526 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2256c75-2461-41d7-b3c5-def733a937e1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.453186 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.453976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.454236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.454717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.463790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.465563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b1a7a-931b-4796-a89e-25a82586e3aa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.481986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1190/2384] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:02:45.483216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.494735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5aad8-beb4-4951-8167-257521f27567 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.499945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.500603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.500887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.501458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] PATH_INFO: `/v3/projects/208c5613a4d940ad8d7c74cb7659fce3/users/1b11ebe27f5945479817c06b3c71c465/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.522072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=208c5613a4d940ad8d7c74cb7659fce3, user_id=1b11ebe27f5945479817c06b3c71c465, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.525465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.561532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1195/2385] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:45 2022] PUT /identity/v3/projects/208c5613a4d940ad8d7c74cb7659fce3/users/1b11ebe27f5945479817c06b3c71c465/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:45.563273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.563556 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.563826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.598338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] MFA Rules not processed for user `1b11ebe27f5945479817c06b3c71c465`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:45.600009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:45.673387 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1191/2386] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:45.677899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.742522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409e9c-cadc-4e30-8385-6ba77d11c883 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.747599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.748430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.748851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.749282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.750163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.752396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.780475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1196/2387] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:45.784999 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.790861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:45.797111 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462ac2-371c-451f-acb3-d3d7d5f5d7fb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.880444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.881061 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.881342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.881736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.882589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.885110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.914510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1192/2388] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:02:45 2022] POST /identity/v3/users => generated 285 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:02:45.920007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.934035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696259-7345-4f33-aa3d-71958becd307 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.939403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.940011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.940290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.940559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.941203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.942792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:45.954788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1197/2389] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:45.958781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:45.970852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f5a4-0251-4e25-8ce9-3593a0095cce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:45.975485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:45.976141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:45.976542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:45.976827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] PATH_INFO: `/v3/projects/eef95c14bd8545ccbf675cda14adbe99/users/f5ec756ae5bb4498bb17845021835950/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:45.992889 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef95c14bd8545ccbf675cda14adbe99, user_id=f5ec756ae5bb4498bb17845021835950, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:45.996216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:46.011686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1193/2390] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:45 2022] PUT /identity/v3/projects/eef95c14bd8545ccbf675cda14adbe99/users/f5ec756ae5bb4498bb17845021835950/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:46.015785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:46.027190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05092d2-1828-48da-8eec-9ac28198fb10 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:46.090996 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:46.091633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:46.091927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:46.092199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:46.092838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:46.094460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c3f1b-bddd-4fa8-adbd-d3ced78f3ea7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:46.105285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1198/2391] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:02:46.109612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:46.121283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3510b3f-1ff1-4606-813d-6f5705c70e3a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:46.125840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:46.126754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:46.127081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:46.127363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] PATH_INFO: `/v3/projects/eef95c14bd8545ccbf675cda14adbe99/users/f5ec756ae5bb4498bb17845021835950/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:46.129254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef95c14bd8545ccbf675cda14adbe99, user_id=f5ec756ae5bb4498bb17845021835950, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:46.132073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:46.143550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1194/2392] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:02:46 2022] PUT /identity/v3/projects/eef95c14bd8545ccbf675cda14adbe99/users/f5ec756ae5bb4498bb17845021835950/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:02:46.154315 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:46.154599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:46.154871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:46.183784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] MFA Rules not processed for user `f5ec756ae5bb4498bb17845021835950`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:46.184982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:46.225945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:46.264644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1199/2393] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:02:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:46.280489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:46.282290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:46.354560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47796c95-923f-4dbd-bce7-3f375b000a16 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:46.358249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:46.358856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:46.359169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:46.359444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:46.368910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:46.370527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ea9fd-1dd6-4cc1-90a5-4bb7680f7a75 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:46.381335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1195/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:46.464653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:46.474703 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da0af0-c4c4-4883-8550-e8465c91b1ec None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:46.479207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:46.479905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:46.480277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:46.480614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:46.546351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:46.547921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:46.558137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1200/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:02:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:02:56.376460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:56.553273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=41 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:02:57.800240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:57.800738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:57.800854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:57.822612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:02:57.824433 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:57.879099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1196/2396] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 18:02:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:02:57.891791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:02:57.893193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:02:57.961979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bb287-155f-4e6b-8d42-4a969716a14d service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:02:57.967640 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:02:57.968244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:02:57.968524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:02:57.968795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:02:57.977171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:02:57.978765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f1bff1-c482-4965-af6c-3b6ffe3f8065 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:02:57.986717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1201/2397] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:02:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:03:01.740135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:01.811483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55508e5-3de1-4000-b4c6-141d11d69e68 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:01.816153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:01.816689 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:01.816966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:01.817236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] PATH_INFO: `/v3/users/1b11ebe27f5945479817c06b3c71c465` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:01.818357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b11ebe27f5945479817c06b3c71c465)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:01.820328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:01.871827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b11ebe27f5945479817c06b3c71c465'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:01.879059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b11ebe27f5945479817c06b3c71c465'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:01.879953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b11ebe27f5945479817c06b3c71c465'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:01.892316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b11ebe27f5945479817c06b3c71c465'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:01.899033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b11ebe27f5945479817c06b3c71c465'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:01.905665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1197/2398] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:01 2022] DELETE /identity/v3/users/1b11ebe27f5945479817c06b3c71c465 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:01.910012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:01.910493 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:01.910616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:01.929714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:01.930891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:01.988954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1202/2399] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:01.999847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:02.001509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:02.028981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:02.059877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288ea4b-c98a-4eb5-a7a0-8dc426b0a58e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:02.063544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:02.064147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:02.064424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:02.064691 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:02.074249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:02.075735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:02.085068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1198/2400] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:02.355755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:02.357202 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:02.368961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3aac9-53ca-4a67-9c4d-cc2e1f3e31cf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:02.373524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:02.374191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:02.374473 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:02.374748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] PATH_INFO: `/v3/projects/208c5613a4d940ad8d7c74cb7659fce3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:02.375951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=208c5613a4d940ad8d7c74cb7659fce3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:02.378168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:02.427149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '208c5613a4d940ad8d7c74cb7659fce3', 'request_id': 'req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.433530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '208c5613a4d940ad8d7c74cb7659fce3', 'request_id': 'req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.439933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '208c5613a4d940ad8d7c74cb7659fce3', 'request_id': 'req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.453540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] The token cache is being invalidate because project 208c5613a4d940ad8d7c74cb7659fce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:02.453826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.454447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.455953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1203/2401] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:02 2022] DELETE /identity/v3/projects/208c5613a4d940ad8d7c74cb7659fce3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:02.459951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:02.470100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d259-053e-4bbb-abcd-cfa627029836 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:02.534643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:02.535214 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:02.535497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:02.535771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] PATH_INFO: `/v3/users/f5ec756ae5bb4498bb17845021835950` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:02.536899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ec756ae5bb4498bb17845021835950)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:02.538815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:02.584986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ec756ae5bb4498bb17845021835950'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.592422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ec756ae5bb4498bb17845021835950'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.593029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ec756ae5bb4498bb17845021835950'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.604543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ec756ae5bb4498bb17845021835950'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.611390 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0dd81b99-76c0-4280-b14b-bcbd867ba21a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ec756ae5bb4498bb17845021835950'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:02.617227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1199/2402] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:02 2022] DELETE /identity/v3/users/f5ec756ae5bb4498bb17845021835950 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:02.901083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:02.903178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:02.919052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1914f6d-7f8e-42dd-9f80-15eafd4eb041 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:02.930992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:02.983437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:02.984092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:02.984374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:02.984646 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] PATH_INFO: `/v3/projects/eef95c14bd8545ccbf675cda14adbe99` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:02.985782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef95c14bd8545ccbf675cda14adbe99)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:02.987711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.037142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef95c14bd8545ccbf675cda14adbe99', 'request_id': 'req-1fb5a103-bc7b-4a6e-a913-08a69c595b88'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:03.042847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef95c14bd8545ccbf675cda14adbe99', 'request_id': 'req-1fb5a103-bc7b-4a6e-a913-08a69c595b88'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:03.049924 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef95c14bd8545ccbf675cda14adbe99', 'request_id': 'req-1fb5a103-bc7b-4a6e-a913-08a69c595b88'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:03.063910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] The token cache is being invalidate because project eef95c14bd8545ccbf675cda14adbe99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:03.064272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:03.064928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-1fb5a103-bc7b-4a6e-a913-08a69c595b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:03.071398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1204/2403] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:02 2022] DELETE /identity/v3/projects/eef95c14bd8545ccbf675cda14adbe99 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:03.077221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.077652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.077910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.098737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:03.099971 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:03.144927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:03.174087 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1200/2404] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:03.176905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.177263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.177565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.199331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:03.200456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:03.208558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1205/2405] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:03.212967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.222642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80200ce-e59f-4a35-96a3-ef04635b19e5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.227485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.228082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.228363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.228664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.229318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:03.229619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.231756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.243243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1201/2406] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:03.248159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.257502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e26d20-2a69-422f-959b-ec1bbf35c80f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.262360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.262956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.263271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.263544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.264163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.266083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.294045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1206/2407] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:03.298406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.308504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dbd6b-4cf0-4a20-82af-f551ca2cdb10 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.369711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.370308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.370588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.370859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.371537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.373460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.397066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1202/2408] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:03.398205 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.408873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf2e6e-2486-467b-baeb-7bca68349e08 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.413381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.413968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.414246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.414517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.415212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.416825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.427258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1207/2409] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:03.431673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.441950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfef01-af97-4663-92f4-c2922131d503 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.446806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.447429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.447707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.447978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.448619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.450207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ed125-850c-41cd-977e-06faa85b5e1f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.460713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1203/2410] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:03.464850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.475569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05e393-d966-45ea-9a59-3568cd713ffe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.479882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.480511 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.481418 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.481696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] PATH_INFO: `/v3/projects/072318c310234fa68f6e8323186786b9/users/dd59e10dee774f1a87bb7d62ad7769c8/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.496173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=072318c310234fa68f6e8323186786b9, user_id=dd59e10dee774f1a87bb7d62ad7769c8, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.498828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.511875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1208/2411] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:03 2022] PUT /identity/v3/projects/072318c310234fa68f6e8323186786b9/users/dd59e10dee774f1a87bb7d62ad7769c8/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:03.521941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.522227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.522583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.550504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] MFA Rules not processed for user `dd59e10dee774f1a87bb7d62ad7769c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:03.551728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:03.628531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1204/2412] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:03.633076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.676075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:03.696303 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb88876-0167-4374-9996-e5a0c5961831 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.701407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.701999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.702273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.702540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.703185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.705065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.732979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1209/2413] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:03.736859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.747600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bf58c-cfa6-435f-be38-6ea4484c8151 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.788906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:03.807683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.808291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.808574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.808857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.809506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.811767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.833074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1205/2414] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:03 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:03.837177 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.848110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10e26c-c2bc-4b76-8adc-bba8c33d84cf admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.852684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.853272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.853552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.853823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.854455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.856129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.867349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1210/2415] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:03.871505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.881665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa833a6-32d9-49cd-90b8-6d7d97564bd0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.886293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.887459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.887761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.888033 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] PATH_INFO: `/v3/projects/3a5b500c842d48a493cda1cbe82f2e46/users/b1ba7d39a13f4d429df7acae69de974e/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.903407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5b500c842d48a493cda1cbe82f2e46, user_id=b1ba7d39a13f4d429df7acae69de974e, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.906254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:03.918500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1206/2416] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:03 2022] PUT /identity/v3/projects/3a5b500c842d48a493cda1cbe82f2e46/users/b1ba7d39a13f4d429df7acae69de974e/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:03.922522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:03.932304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b52d2e-24cf-4555-b17b-381ecfbd3ef3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:03.987130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:03.987717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:03.987993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:03.988261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:03.988888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:03.990536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7d52b-e28f-4cda-8bb8-9b050f87b18a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:04.000151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1211/2417] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:04.004653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:04.016118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3c87-daf9-47cd-9ddb-a965d3f43cd4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:04.020766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:04.021421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.021696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.021966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] PATH_INFO: `/v3/projects/3a5b500c842d48a493cda1cbe82f2e46/users/b1ba7d39a13f4d429df7acae69de974e/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.023896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5b500c842d48a493cda1cbe82f2e46, user_id=b1ba7d39a13f4d429df7acae69de974e, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:04.026517 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:04.037163 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1207/2418] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:04 2022] PUT /identity/v3/projects/3a5b500c842d48a493cda1cbe82f2e46/users/b1ba7d39a13f4d429df7acae69de974e/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:04.048288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.048573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.048844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.078236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] MFA Rules not processed for user `b1ba7d39a13f4d429df7acae69de974e`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:04.079506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:04.147661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1212/2419] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:04.159083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:04.160782 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:04.218709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f24e22-109c-4c4e-a566-4105082593b0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:04.222507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:04.223165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.223452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.223729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.233168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:04.234582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:04.244090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1208/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:04.338962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:04.348713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d111f76-62ce-41c9-a6b5-8838da1fbe13 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:04.356634 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:04.357238 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.357513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.357779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.392016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:04.416762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:04.418121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10648f41-69a5-45cc-b0d2-6168884ecdca service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:04.426830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1213/2421] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:04.689354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:04.752284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe798d75-b8c9-4b9c-8a42-dba8009d1d3c service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:04.756796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:04.757415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.757692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.757962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] PATH_INFO: `/v3/users/dd59e10dee774f1a87bb7d62ad7769c8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.759147 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd59e10dee774f1a87bb7d62ad7769c8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:04.761044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:04.801640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:04.809722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd59e10dee774f1a87bb7d62ad7769c8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:04.817652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd59e10dee774f1a87bb7d62ad7769c8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:04.818268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd59e10dee774f1a87bb7d62ad7769c8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:04.830017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd59e10dee774f1a87bb7d62ad7769c8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:04.836756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd59e10dee774f1a87bb7d62ad7769c8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:04.843093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1209/2422] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:04 2022] DELETE /identity/v3/users/dd59e10dee774f1a87bb7d62ad7769c8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:04.847609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.847903 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.848182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:04.866957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:04.868170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:04.925413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1214/2423] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:04.935045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:04.936432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:04.994755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8944-e6fa-4cc4-ab02-2cab318cf9c6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:04.998474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:04.999110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:04.999391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:04.999659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:05.008988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:05.010416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:05.019761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1210/2424] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:05.302734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:05.304535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:05.317945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06043b3-7853-4aa6-9541-76e13e4083ee None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:05.322802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:05.323309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:05.323804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:05.324120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] PATH_INFO: `/v3/projects/072318c310234fa68f6e8323186786b9` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:05.325906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=072318c310234fa68f6e8323186786b9)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:05.327954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:05.375450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072318c310234fa68f6e8323186786b9', 'request_id': 'req-a00d230e-d81e-4ced-be1e-05356040cbdb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.381099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072318c310234fa68f6e8323186786b9', 'request_id': 'req-a00d230e-d81e-4ced-be1e-05356040cbdb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.390412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072318c310234fa68f6e8323186786b9', 'request_id': 'req-a00d230e-d81e-4ced-be1e-05356040cbdb'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.403647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] The token cache is being invalidate because project 072318c310234fa68f6e8323186786b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:05.403938 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.405420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.407361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1215/2425] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:05 2022] DELETE /identity/v3/projects/072318c310234fa68f6e8323186786b9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:05.410837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:05.420567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517e4ec-743d-4ef2-a43e-2210df32c025 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:05.476635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:05.477254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:05.477528 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:05.477795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] PATH_INFO: `/v3/users/b1ba7d39a13f4d429df7acae69de974e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:05.478890 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ba7d39a13f4d429df7acae69de974e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:05.480853 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:05.526054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ba7d39a13f4d429df7acae69de974e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.533721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ba7d39a13f4d429df7acae69de974e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.534421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ba7d39a13f4d429df7acae69de974e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.539491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:05.547730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ba7d39a13f4d429df7acae69de974e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.555059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2587155c-c79e-4457-b5b5-a8bd0abe10be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ba7d39a13f4d429df7acae69de974e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.561348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1211/2426] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:05 2022] DELETE /identity/v3/users/b1ba7d39a13f4d429df7acae69de974e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:05.860573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:05.862042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:05.875561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d230e-d81e-4ced-be1e-05356040cbdb admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:05.932292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:05.939195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:05.939827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:05.940398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:05.940725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] PATH_INFO: `/v3/projects/3a5b500c842d48a493cda1cbe82f2e46` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:05.942076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5b500c842d48a493cda1cbe82f2e46)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:05.944358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:05.991233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5b500c842d48a493cda1cbe82f2e46', 'request_id': 'req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:05.997847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5b500c842d48a493cda1cbe82f2e46', 'request_id': 'req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:06.004371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5b500c842d48a493cda1cbe82f2e46', 'request_id': 'req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:06.018150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] The token cache is being invalidate because project 3a5b500c842d48a493cda1cbe82f2e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:06.018441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:06.019122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-3b98cbf5-6099-4db2-9837-c4e82b3c7dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:06.020604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1216/2427] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:05 2022] DELETE /identity/v3/projects/3a5b500c842d48a493cda1cbe82f2e46 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:06.025507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.025838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.026151 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.045881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:06.047137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:06.108510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1212/2428] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:06.116536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.116818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.117093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.137840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:06.139242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:06.147560 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1217/2429] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:06.151449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.161238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92f95-ea0c-4d42-8dd1-bbab641d16a7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.165918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.166507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.166789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.167104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.167748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:06.168050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.170162 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.181552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1213/2430] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:06.186545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.196112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cdf40-1413-467c-b08d-891606bf1e0a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.200670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.201252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.201524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.201790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.202515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.204414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.231521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1218/2431] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:06 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:06.235707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.245673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd2ed-2acc-4c5a-9c98-86512b4e79d0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.302245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.302847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.303185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.303457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.304100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.306066 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.327352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1214/2432] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:06 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:06.330763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.341908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810e4fc-be84-4783-ad60-7387fc41e0fe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.346510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.347149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.347433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.347706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.348360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.349940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.360785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1219/2433] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:06.364833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.375191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff46b-bdc3-42a1-bc03-2a082c83792d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.379640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.380246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.380525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.380794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.381432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.383031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ca0e-99ad-4811-8543-70b2e80e6096 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.393502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1215/2434] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:06.397684 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.408088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f62b6d-b10f-4e92-98c1-fb2ef3eec9c4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.412689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.413332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.413615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.413886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] PATH_INFO: `/v3/projects/26f43f9c051f40f2a208183bc5e14698/users/f580181bfbf34f8fa18c2097f4ce9ec3/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.429804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f43f9c051f40f2a208183bc5e14698, user_id=f580181bfbf34f8fa18c2097f4ce9ec3, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.432501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.446086 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1220/2435] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:06 2022] PUT /identity/v3/projects/26f43f9c051f40f2a208183bc5e14698/users/f580181bfbf34f8fa18c2097f4ce9ec3/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:06.457313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10491313-840d-4bd0-ae77-2af366eb562d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.457775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10491313-840d-4bd0-ae77-2af366eb562d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.457871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10491313-840d-4bd0-ae77-2af366eb562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.488606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-10491313-840d-4bd0-ae77-2af366eb562d None None] MFA Rules not processed for user `f580181bfbf34f8fa18c2097f4ce9ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:06.489829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-10491313-840d-4bd0-ae77-2af366eb562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:06.525423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:06.567008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1216/2436] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:06.577685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:06.579655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:06.642088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e0fc2-c760-49d4-86ad-e22a9a943987 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:06.645675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:06.646270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:06.646544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:06.646863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:06.656326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:06.657743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb4529-75eb-43f4-8ced-d7146b418ff7 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:06.666628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1221/2437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:08.041794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:08.042249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:08.042375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:08.063194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] MFA Rules not processed for user `f580181bfbf34f8fa18c2097f4ce9ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:08.064135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:08.071637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1217/2438] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:16.662697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:18.056857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=15 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:27.749203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47671eb-3a1b-46f5-86b1-7bd0eed6e6d0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:27.749812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47671eb-3a1b-46f5-86b1-7bd0eed6e6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:27.749920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47671eb-3a1b-46f5-86b1-7bd0eed6e6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:27.772969 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b47671eb-3a1b-46f5-86b1-7bd0eed6e6d0 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:27.774313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b47671eb-3a1b-46f5-86b1-7bd0eed6e6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:27.849091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1222/2439] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:03:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:27.871725 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:27.873924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:27.956838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f985bc-d692-4382-b1ca-eab78610ee5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:27.963540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:27.964644 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:27.964984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:27.965431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:27.976424 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:27.978119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:27.988491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1218/2440] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:03:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:03:28.729110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e7b-bd7c-40c0-a086-4b1edb7e8e1d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:28.729329 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e7b-bd7c-40c0-a086-4b1edb7e8e1d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:28.729607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48730e7b-bd7c-40c0-a086-4b1edb7e8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:28.750047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-48730e7b-bd7c-40c0-a086-4b1edb7e8e1d None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:28.751644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-48730e7b-bd7c-40c0-a086-4b1edb7e8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:28.761921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1223/2441] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 18:03:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:28.776306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:28.787350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19747680-764d-4861-aaf0-bb62b8fb2cdf service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:28.794121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:28.794800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:28.795137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:28.795418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:28.806561 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:28.808622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:28.820532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1219/2442] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:03:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:03:29.101562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2578d4-5257-448b-98e5-16432ec4868c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:29.101843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2578d4-5257-448b-98e5-16432ec4868c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:29.102117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2578d4-5257-448b-98e5-16432ec4868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:29.126228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7b2578d4-5257-448b-98e5-16432ec4868c None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:29.127688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7b2578d4-5257-448b-98e5-16432ec4868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:29.135184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1224/2443] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:29.147963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:29.158675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d14ab2-ee9f-44e8-8de8-eda2cc5b4f56 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:29.164030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:29.164748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:29.165097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:29.165436 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:29.174918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:29.176648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:29.186309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1220/2444] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:03:29.522559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:29.522851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:29.523166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:29.542276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:29.543596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:29.550668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1225/2445] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:29.563645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:29.574392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ed9ad-7457-4bf3-b561-e35600da81e5 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:29.580126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:29.580783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:29.581113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:29.581384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:29.589932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:29.592122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea0d69-503e-4cd1-a142-66789aba76c2 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:29.600648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1221/2446] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:03:31.931973 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:32.018814 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85bb2d-3284-438a-9652-93294188bab7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:32.025400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:32.026006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:32.026481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:32.026768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] PATH_INFO: `/v3/users/f580181bfbf34f8fa18c2097f4ce9ec3` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:32.028592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f580181bfbf34f8fa18c2097f4ce9ec3)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:32.030756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:32.112653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f580181bfbf34f8fa18c2097f4ce9ec3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.143597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f580181bfbf34f8fa18c2097f4ce9ec3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.145472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f580181bfbf34f8fa18c2097f4ce9ec3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.167021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f580181bfbf34f8fa18c2097f4ce9ec3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.184797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f580181bfbf34f8fa18c2097f4ce9ec3'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.196486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1226/2447] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:31 2022] DELETE /identity/v3/users/f580181bfbf34f8fa18c2097f4ce9ec3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:32.197696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:32.198145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:32.198242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:32.233652 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:32.235554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:32.334930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1222/2448] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:32.341603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:32.343033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:32.351902 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:32.414769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4947b1-b62d-4fe2-9feb-362a6a77f381 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:32.419872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82342856-143b-4a8c-821b-815688775ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:32.420871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342856-143b-4a8c-821b-815688775ded service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:32.420871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342856-143b-4a8c-821b-815688775ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:32.421200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342856-143b-4a8c-821b-815688775ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:32.432800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342856-143b-4a8c-821b-815688775ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:32.436226 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342856-143b-4a8c-821b-815688775ded service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:32.446517 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1227/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:32.757743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:32.758717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:32.774448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d85bb6-a6ae-428a-9ae3-5db2377063ca None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:32.780757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:32.781398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:32.781866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:32.782296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] PATH_INFO: `/v3/projects/26f43f9c051f40f2a208183bc5e14698` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:32.783650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f43f9c051f40f2a208183bc5e14698)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:32.787586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:32.851521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:32.854939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f43f9c051f40f2a208183bc5e14698', 'request_id': 'req-5d05da7b-8ec8-452d-81cc-80948b6a0359'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.868776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f43f9c051f40f2a208183bc5e14698', 'request_id': 'req-5d05da7b-8ec8-452d-81cc-80948b6a0359'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.877532 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f43f9c051f40f2a208183bc5e14698', 'request_id': 'req-5d05da7b-8ec8-452d-81cc-80948b6a0359'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.890856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] The token cache is being invalidate because project 26f43f9c051f40f2a208183bc5e14698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:32.891472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.892267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5d05da7b-8ec8-452d-81cc-80948b6a0359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:32.894144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1223/2450] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:32 2022] DELETE /identity/v3/projects/26f43f9c051f40f2a208183bc5e14698 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:32.898879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:32.899233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:32.899453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:32.926431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:32.927723 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:32.995490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1228/2451] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:33.005026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.005337 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.005682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.026072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:33.027361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:33.035868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1224/2452] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:33.040430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.052480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff2303-aedd-4d6d-8df6-2a5b1729dddf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.057300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.057918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.058231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.058503 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.059232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:33.059541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.061138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.075989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1229/2453] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:33.080334 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.089827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e2a4-41de-450a-85a5-e56819c37907 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.095761 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.096382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.096665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.096941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.097572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.100906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.129012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1225/2454] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:33 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:33.133447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.144574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacf243-ae86-4044-9044-5fb575164594 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.208156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.208744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.209021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.209487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.210131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.212157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.236293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1230/2455] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:33 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:33.240534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.250946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ee83-7dc4-4e89-a5ce-9926a7b59c46 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.256794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.257395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.257671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.257938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.258580 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.260222 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.271414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1226/2456] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:33.275878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.286488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd380eb-444f-46df-8a5e-d239c91b32aa admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.291707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.292312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.292595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.292914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.293505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.295848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c332db8-932a-4729-9650-04c64b5848c2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.307071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1231/2457] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:33.312005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.322810 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd30fa1-9589-46fc-ba6f-666e63f7eaa1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.328763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.329430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.329791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.329960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] PATH_INFO: `/v3/projects/fae6073847f2416da72f6c9276da44cf/users/842b7f1ea05f41d0b000f97ee9bf7196/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.345304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae6073847f2416da72f6c9276da44cf, user_id=842b7f1ea05f41d0b000f97ee9bf7196, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.348019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.360695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1227/2458] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:33 2022] PUT /identity/v3/projects/fae6073847f2416da72f6c9276da44cf/users/842b7f1ea05f41d0b000f97ee9bf7196/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:33.365788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdca1c-8d86-463c-b2c2-e766ea34f55a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.366072 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdca1c-8d86-463c-b2c2-e766ea34f55a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.366343 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdca1c-8d86-463c-b2c2-e766ea34f55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.385382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-a4cdca1c-8d86-463c-b2c2-e766ea34f55a None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:33.386574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a4cdca1c-8d86-463c-b2c2-e766ea34f55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:33.395185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:33.445270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1232/2459] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:33.508980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:33.510811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:33.568986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335aca8-b997-4f33-b551-320542c50968 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:33.572735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:33.573343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:33.573621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:33.573895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:33.582988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:33.584466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:33.593771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1228/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:34.581586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3bbfb-4d45-41c3-bab0-d49c439ea7b0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:34.582057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3bbfb-4d45-41c3-bab0-d49c439ea7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:34.582463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3bbfb-4d45-41c3-bab0-d49c439ea7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:34.614305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-61a3bbfb-4d45-41c3-bab0-d49c439ea7b0 None None] MFA Rules not processed for user `842b7f1ea05f41d0b000f97ee9bf7196`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:34.616363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-61a3bbfb-4d45-41c3-bab0-d49c439ea7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:34.708289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1233/2461] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:34.719801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:34.724924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:34.796707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da11b1-bb8c-45f6-b06d-a50d1a16ae3e service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:34.800386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:34.800990 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:34.801272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:34.801539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:34.811004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:34.812450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:34.821969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1229/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:36.181668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:36.182165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:36.182264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:36.200567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] MFA Rules not processed for user `842b7f1ea05f41d0b000f97ee9bf7196`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:36.201788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:36.208659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1234/2463] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:36.440923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:36.442322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:36.454670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406152fa-b871-42e7-8e2e-f880e6e9b1b1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:36.458233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:36.458820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:36.459139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:36.459409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:36.461070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:36.470949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:36.472411 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:36.480752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1230/2464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:36.581846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:36.591659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c79fb0-a151-4db4-82ba-f00a13a5a4cb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:36.595129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:36.595721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:36.595994 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:36.596261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:36.597946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:36.610745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:36.612211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:36.622581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1235/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:38.279352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:38.283646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:38.288897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb190-b6bd-4c9b-876e-900d2d83ec2e service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:38.294936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:38.295998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:38.296670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:38.296968 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:38.299041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:38.311628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:38.313049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:38.322939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1231/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:41.670851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:41.685039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:41.758539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:41.761509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:41.790021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e78932-80f5-42e9-a0ab-6e6cda1d5de5 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:41.796229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:41.797045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:41.797484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:41.797936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:41.799544 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1497e-3d2f-447f-a472-45aaf12100c9 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:41.800133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:41.816021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:41.817984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:41.820558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:41.821220 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:41.822700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:41.823048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] PATH_INFO: `/v3/users/478a0e3ac431440582d3dca1fe246ac3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:41.824308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478a0e3ac431440582d3dca1fe246ac3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:41.828267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:41.831975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1236/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:41.936005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478a0e3ac431440582d3dca1fe246ac3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:41.951865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478a0e3ac431440582d3dca1fe246ac3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:41.953373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478a0e3ac431440582d3dca1fe246ac3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:41.975464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478a0e3ac431440582d3dca1fe246ac3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:41.988165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-692f2cb6-fc9a-4c79-8282-3348eb265f19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478a0e3ac431440582d3dca1fe246ac3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.002686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1232/2468] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:41 2022] DELETE /identity/v3/users/478a0e3ac431440582d3dca1fe246ac3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:42.343756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:42.357108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bad09-804f-41b4-909f-814c527f6d21 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:42.368614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:42.439842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:42.440769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.441210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.441489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] PATH_INFO: `/v3/projects/f9182ab0c79d4b8a9de4c319491ca316` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.442719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9182ab0c79d4b8a9de4c319491ca316)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:42.444639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:42.499540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9182ab0c79d4b8a9de4c319491ca316', 'request_id': 'req-871058f9-3a32-466b-8c9d-23d16ae2375f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.506148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9182ab0c79d4b8a9de4c319491ca316', 'request_id': 'req-871058f9-3a32-466b-8c9d-23d16ae2375f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.514055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9182ab0c79d4b8a9de4c319491ca316', 'request_id': 'req-871058f9-3a32-466b-8c9d-23d16ae2375f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.543675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] The token cache is being invalidate because project f9182ab0c79d4b8a9de4c319491ca316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:42.544185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.544971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.546752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1237/2469] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:42 2022] DELETE /identity/v3/projects/f9182ab0c79d4b8a9de4c319491ca316 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:42.552585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668386-a92a-4454-b1d5-2cbf0efff9e4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.552868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668386-a92a-4454-b1d5-2cbf0efff9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.553135 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90668386-a92a-4454-b1d5-2cbf0efff9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.577197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-90668386-a92a-4454-b1d5-2cbf0efff9e4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:42.578472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90668386-a92a-4454-b1d5-2cbf0efff9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:42.650908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1233/2470] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:42.657142 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:42.659256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.659547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.659825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.660159 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:42.678513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871058f9-3a32-466b-8c9d-23d16ae2375f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:42.681041 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:42.682332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:42.684927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:42.685815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.686308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.686575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] PATH_INFO: `/v3/users/cb11c04acb4f40cd9d09edf16b11abb0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.687993 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb11c04acb4f40cd9d09edf16b11abb0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:42.690407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:42.695603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1234/2471] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:42.696448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:42.709072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d448fb-bd4e-4600-9a23-cbd8e8b597e6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:42.720963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:42.721708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.722118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.722427 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.723386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:42.723668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:42.725843 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:42.747143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1235/2472] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:42.748541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:42.754707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb11c04acb4f40cd9d09edf16b11abb0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.760569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d25e1-da2c-4d20-ab40-fcfb2fa029c2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:42.762424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb11c04acb4f40cd9d09edf16b11abb0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.763145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb11c04acb4f40cd9d09edf16b11abb0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.765530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:42.766126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.766476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.766677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.767418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:42.769549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:42.773751 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb11c04acb4f40cd9d09edf16b11abb0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.792197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb11c04acb4f40cd9d09edf16b11abb0'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:42.802469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1238/2473] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:42 2022] DELETE /identity/v3/users/cb11c04acb4f40cd9d09edf16b11abb0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:42.805784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1236/2474] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:42 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:42.810551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:42.812010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:42.832231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:42.887427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09065591-e1a1-40dd-bdfe-c3fc51f69496 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:42.963983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:42.964710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:42.965088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:42.966038 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:42.969393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:42.969393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:42.995034 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1239/2475] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:42 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:42.999491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.012973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee775bb8-6b7d-44f4-816e-ac041b86e4cb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.019535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.020149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.020627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.021094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.022051 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.024395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.036875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1237/2476] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:43.041089 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.052984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94e74-8467-4850-b988-d06c2daa5e35 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.061368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.063148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.073416 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.074357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.075220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.075708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.076428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.078093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.080384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f4dc4-1e89-4020-a08b-55021c6ae327 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.085963 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.087071 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.087587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.088081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] PATH_INFO: `/v3/projects/6816355a42d64d5988aff83532ff5888` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.089688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6816355a42d64d5988aff83532ff5888)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.092137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.095778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:43.098131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1240/2477] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:43.102158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.114940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6e4b-8ace-48ec-9a18-5d99405cd89d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.121125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.122311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.122612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.122878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] PATH_INFO: `/v3/projects/8bc6913abd5546fe9158612fe56a5f1c/users/5b2b570130da45aaa2ed28340bd6cd96/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.142836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc6913abd5546fe9158612fe56a5f1c, user_id=5b2b570130da45aaa2ed28340bd6cd96, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.145970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75c3e-e4f0-486d-b0e5-4743ef516a40 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.157159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6816355a42d64d5988aff83532ff5888', 'request_id': 'req-e4d3ed01-4da9-4c66-905b-07ae00282d98'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:43.163476 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1241/2478] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:43 2022] PUT /identity/v3/projects/8bc6913abd5546fe9158612fe56a5f1c/users/5b2b570130da45aaa2ed28340bd6cd96/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:43.171587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6816355a42d64d5988aff83532ff5888', 'request_id': 'req-e4d3ed01-4da9-4c66-905b-07ae00282d98'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:43.174554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.174842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.175151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.180198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6816355a42d64d5988aff83532ff5888', 'request_id': 'req-e4d3ed01-4da9-4c66-905b-07ae00282d98'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:43.192718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] The token cache is being invalidate because project 6816355a42d64d5988aff83532ff5888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:43.192979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:43.193805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e4d3ed01-4da9-4c66-905b-07ae00282d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:43.195616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1238/2479] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:43 2022] DELETE /identity/v3/projects/6816355a42d64d5988aff83532ff5888 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:43.202152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfe9b-6c8c-48e9-a371-482badb8c887 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.202446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfe9b-6c8c-48e9-a371-482badb8c887 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.202797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62dfe9b-6c8c-48e9-a371-482badb8c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.211137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] MFA Rules not processed for user `5b2b570130da45aaa2ed28340bd6cd96`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:43.212554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.230502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d62dfe9b-6c8c-48e9-a371-482badb8c887 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:43.231985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d62dfe9b-6c8c-48e9-a371-482badb8c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.300673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1242/2480] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:43.304465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1239/2481] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:43.309061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.310714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.311601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.311891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.312166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.322622 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:43.345165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:43.346885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.359809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1240/2482] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:43.363732 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.374423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db0793-97b9-4844-9e5e-5b963151689c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.380150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.380849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.381131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.381404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.382053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:43.382353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.383988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.397007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1241/2483] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:03:43.402075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.410132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420241-8fc4-4216-b151-c480f59bbc01 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.415338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba0370-6f63-45b0-8d9b-c8ea4f0762ae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.416104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.416927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.417488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.417824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.420475 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.421998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.421998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.421998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.422643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.424935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.427080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.428590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.439465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1243/2484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:43.459064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1242/2485] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:43.464957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.476964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eb2e1-4b97-4034-b361-e41f2349288f service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.539367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.539967 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.540246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.540537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.541168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.543348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.572879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1244/2486] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:43.577665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.590032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a7d448-4a8d-4e95-99d2-98efe40fcc87 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.596114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.596114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.597010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.597362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.598070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.599765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.610785 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1243/2487] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:43.615079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.625508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b151371-77e3-42e1-94fa-9c39aee8aea4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.631557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.632788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.632788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.632788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.633310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.635242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8bc48-7cd5-468c-807b-2853d07d2123 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.647502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1245/2488] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:43.652152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.669822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234afda-1c15-490e-b537-801328487f03 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.677367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.678110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.678478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.681115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] PATH_INFO: `/v3/projects/9134f4ef5192451b96f8ac427e3514e7/users/c7cca4ae6f2f43c88d5269a74a6b52ea/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.701183 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9134f4ef5192451b96f8ac427e3514e7, user_id=c7cca4ae6f2f43c88d5269a74a6b52ea, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.706550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:43.724414 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1244/2489] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:43 2022] PUT /identity/v3/projects/9134f4ef5192451b96f8ac427e3514e7/users/c7cca4ae6f2f43c88d5269a74a6b52ea/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:43.736887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.737452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.737452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.774387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] MFA Rules not processed for user `c7cca4ae6f2f43c88d5269a74a6b52ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:43.775689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:43.878028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:43.882122 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1246/2490] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:43.890711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:43.920137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:43.987326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5bf7d-3317-4b13-bebb-21b5c8285e2b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:43.992811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:43.993428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:43.993715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:43.993981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:43.994893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:43.997378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.030032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1245/2491] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:43 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:44.034171 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.046138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e876-154f-449f-907e-a4e1d07578d6 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.116443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.117051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.117330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.117601 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.118267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.120405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.146928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1247/2492] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:44 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:44.148675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.160100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26653eb-e9e9-4171-83b3-88531301e17d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.166326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.167362 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.167866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.168866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.170054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.173404 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.199323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1246/2493] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:44.200894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.211844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387b07-814a-44b9-ba89-1500a2de7892 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.216821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.217489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.217792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.218071 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] PATH_INFO: `/v3/projects/c59e6f6d23ca482ca4985b2afe044937/users/e7c9c7f20c9d47298bd50e503c88fd0a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.240848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59e6f6d23ca482ca4985b2afe044937, user_id=e7c9c7f20c9d47298bd50e503c88fd0a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.244121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.259015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1248/2494] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:44 2022] PUT /identity/v3/projects/c59e6f6d23ca482ca4985b2afe044937/users/e7c9c7f20c9d47298bd50e503c88fd0a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:44.263873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.279538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b515b-271d-46a9-9d8c-c998e69dd6eb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.360311 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.361332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.361769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.362092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.362948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.365354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a201930-a318-49af-aea7-632579f06e61 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.378301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1247/2495] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:44.382576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.395697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64957e4e-e5db-4541-8add-358b12f9927a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.400548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.401264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.401553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.402039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] PATH_INFO: `/v3/projects/c59e6f6d23ca482ca4985b2afe044937/users/e7c9c7f20c9d47298bd50e503c88fd0a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.404039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c59e6f6d23ca482ca4985b2afe044937, user_id=e7c9c7f20c9d47298bd50e503c88fd0a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.406686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.421804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1249/2496] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:44 2022] PUT /identity/v3/projects/c59e6f6d23ca482ca4985b2afe044937/users/e7c9c7f20c9d47298bd50e503c88fd0a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:44.432217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.432503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.432780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.464945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] MFA Rules not processed for user `e7c9c7f20c9d47298bd50e503c88fd0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:44.466759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:44.541359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1248/2497] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:44.554460 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:44.556225 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:44.631674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f0211-8612-4ce8-9422-311712266d17 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:44.635387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:44.635990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:44.636277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:44.637783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:44.647983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:44.649422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc37b1-3332-4b32-801b-9afa9b4410de service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:44.660757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1250/2498] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:45.133344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:45.146164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:45.202786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed6372-f689-451a-89b7-c5193d4aaa33 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:45.208085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:45.208884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:45.209086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:45.209407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] PATH_INFO: `/v3/users/c7cca4ae6f2f43c88d5269a74a6b52ea` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:45.210533 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7cca4ae6f2f43c88d5269a74a6b52ea)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:45.212577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:45.264828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7cca4ae6f2f43c88d5269a74a6b52ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.272121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7cca4ae6f2f43c88d5269a74a6b52ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.272767 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7cca4ae6f2f43c88d5269a74a6b52ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.284891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7cca4ae6f2f43c88d5269a74a6b52ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.292019 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7cca4ae6f2f43c88d5269a74a6b52ea'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.299243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1249/2499] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:45 2022] DELETE /identity/v3/users/c7cca4ae6f2f43c88d5269a74a6b52ea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:45.303765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:45.304427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:45.304771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:45.328725 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:45.329915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:45.381618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:45.397836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1251/2500] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:45.408105 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:45.409626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:45.471002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f964e0b-b92d-499e-bb3e-5505edc7f76d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:45.474624 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:45.475326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:45.475687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:45.476432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:45.486095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:45.487558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:45.498353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1250/2501] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:45.793017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:45.795950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:45.807912 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8efc8-852a-4194-b48f-75ea9456c005 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:45.813121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:45.813271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:45.813429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:45.813700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] PATH_INFO: `/v3/projects/9134f4ef5192451b96f8ac427e3514e7` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:45.814866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9134f4ef5192451b96f8ac427e3514e7)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:45.816895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:45.890005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9134f4ef5192451b96f8ac427e3514e7', 'request_id': 'req-5835ec24-2960-4842-9e27-8b4b0ede142b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.897099 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9134f4ef5192451b96f8ac427e3514e7', 'request_id': 'req-5835ec24-2960-4842-9e27-8b4b0ede142b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.904435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9134f4ef5192451b96f8ac427e3514e7', 'request_id': 'req-5835ec24-2960-4842-9e27-8b4b0ede142b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.924720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] The token cache is being invalidate because project 9134f4ef5192451b96f8ac427e3514e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:45.925006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.925620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:45.927141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1252/2502] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:45 2022] DELETE /identity/v3/projects/9134f4ef5192451b96f8ac427e3514e7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:45.931438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:45.941075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6baa-0eca-4be6-899c-23cb44ce6c10 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:45.971511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:46.000138 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.000789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.001069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.001344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] PATH_INFO: `/v3/users/e7c9c7f20c9d47298bd50e503c88fd0a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.002727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c9c7f20c9d47298bd50e503c88fd0a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.004701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.050429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c9c7f20c9d47298bd50e503c88fd0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.058098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c9c7f20c9d47298bd50e503c88fd0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.058900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c9c7f20c9d47298bd50e503c88fd0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.072497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c9c7f20c9d47298bd50e503c88fd0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.080596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c9c7f20c9d47298bd50e503c88fd0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.086635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1251/2503] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:45 2022] DELETE /identity/v3/users/e7c9c7f20c9d47298bd50e503c88fd0a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:46.306304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.306304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.320866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835ec24-2960-4842-9e27-8b4b0ede142b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.406289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.407005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.407312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.407643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] PATH_INFO: `/v3/users/5b2b570130da45aaa2ed28340bd6cd96` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.408856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2b570130da45aaa2ed28340bd6cd96)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.411022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.434686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.436757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.468190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e998d9-3c38-4e7c-a19b-e54e0539fe1c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.481921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.482620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.482956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.483084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2b570130da45aaa2ed28340bd6cd96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.483255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] PATH_INFO: `/v3/projects/c59e6f6d23ca482ca4985b2afe044937` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.485491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c59e6f6d23ca482ca4985b2afe044937)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.487483 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.491174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2b570130da45aaa2ed28340bd6cd96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.491873 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2b570130da45aaa2ed28340bd6cd96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.500759 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2b570130da45aaa2ed28340bd6cd96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.518477 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c78734ef-49bb-4e0d-aa66-2511ea888702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2b570130da45aaa2ed28340bd6cd96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.520301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:46.528361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1253/2504] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:46 2022] DELETE /identity/v3/users/5b2b570130da45aaa2ed28340bd6cd96 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:46.533077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.533366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.533639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.555140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c59e6f6d23ca482ca4985b2afe044937', 'request_id': 'req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.555382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:46.556571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.565458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c59e6f6d23ca482ca4985b2afe044937', 'request_id': 'req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.575341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c59e6f6d23ca482ca4985b2afe044937', 'request_id': 'req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.591403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] The token cache is being invalidate because project c59e6f6d23ca482ca4985b2afe044937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:46.591685 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.592363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a9a70e5-821b-4eb4-a0c5-ffa1d029b66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:46.594188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1252/2505] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:46 2022] DELETE /identity/v3/projects/c59e6f6d23ca482ca4985b2afe044937 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:46.600069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.600431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.600770 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.622057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1254/2506] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:46.633306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.634773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.712475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbe349-895f-48db-9e23-1cde9b031e43 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.716325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.717039 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.717320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.717591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.734130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.735970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebdc36-ff0c-45db-ad32-4504233c3cae service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.747906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1255/2507] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:46.760529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:46.761723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.771616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1253/2508] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:46.780020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.780341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.780565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.810386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:46.813152 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:46.827747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1256/2509] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:46.832099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.844988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81321b-023f-4408-898f-d3a85e9faea3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.850722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.851546 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.851945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.852745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.853422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:46.853721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.855465 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.868945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1254/2510] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:03:46.873423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.884174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9f165-8b48-4706-8f23-489dcfc75cc5 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.892611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:46.893804 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:46.894119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:46.894392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:46.895194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:46.897120 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:46.927899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1257/2511] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:46 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:46.931641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:46.944612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c78c4f-c9b7-4dfe-a85d-76b029bac9a8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:46.966219 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:47.034001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.034940 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.035229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.035603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.037371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.042932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.087955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1255/2512] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:46 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:47.088688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.104590 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c7d1-2709-41cf-8863-0b9b1e56b386 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.111350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.112252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.112530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.112800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.113597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.115411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.127398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1258/2513] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:47.131123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.148840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02679c8f-4f45-48fa-a972-2863d20cea22 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.155029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.156010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.156308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.156612 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.157265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.158988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.173537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1256/2514] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:47.174899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.184974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.188191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.191181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faeb81d-0a40-4095-81a1-c672f5ebfb63 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.197763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.197763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.198118 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.198433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] PATH_INFO: `/v3/projects/13321ac853bb4b709a88e2eba6e88d79/users/076f374c1f8041d4b5eadaa2bd9870ff/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.219616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13321ac853bb4b709a88e2eba6e88d79, user_id=076f374c1f8041d4b5eadaa2bd9870ff, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.222480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37780bf2-e8f0-4a03-8a59-3a799937d5b8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.229784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b9416-4102-4fdc-b7ab-e7859ca20be8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.235922 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.236611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.236895 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.237842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] PATH_INFO: `/v3/projects/8bc6913abd5546fe9158612fe56a5f1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.239079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc6913abd5546fe9158612fe56a5f1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.240921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1259/2515] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:47 2022] PUT /identity/v3/projects/13321ac853bb4b709a88e2eba6e88d79/users/076f374c1f8041d4b5eadaa2bd9870ff/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:47.241069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.262644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.262901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.264452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.311097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc6913abd5546fe9158612fe56a5f1c', 'request_id': 'req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:47.320008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] MFA Rules not processed for user `076f374c1f8041d4b5eadaa2bd9870ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:47.320781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc6913abd5546fe9158612fe56a5f1c', 'request_id': 'req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:47.322384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.328833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc6913abd5546fe9158612fe56a5f1c', 'request_id': 'req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:47.341049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] The token cache is being invalidate because project 8bc6913abd5546fe9158612fe56a5f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:47.341341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:47.342008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-93fca9ba-8ca3-4a0b-82f2-cffd064838b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:47.343545 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1257/2516] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:47 2022] DELETE /identity/v3/projects/8bc6913abd5546fe9158612fe56a5f1c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:47.348807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94406-2351-4a83-b0fd-129c4fe093a3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.349120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94406-2351-4a83-b0fd-129c4fe093a3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.349391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd94406-2351-4a83-b0fd-129c4fe093a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.360485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:47.368774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-acd94406-2351-4a83-b0fd-129c4fe093a3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:47.369986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-acd94406-2351-4a83-b0fd-129c4fe093a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.401631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1260/2517] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:47.406811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.408233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.445679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1258/2518] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:47.447236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.447525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.447799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.466953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:47.468173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.476152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1259/2519] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:47.477887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a0764-507f-4030-a7d9-79b562c7ac18 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.480312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.483869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.484617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.485082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.485452 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.486243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.488708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.490711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa846e37-4728-42e3-aa75-6f208263169b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.494935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.495618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.495900 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.496170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.497374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:47.497678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.499295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.506503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:47.514388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1260/2520] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:03:47.518719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.529332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1261/2521] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:47.530729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4352e4b-53f1-4c14-8f5a-f71b3f57091b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.533513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.593168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.593773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.594048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.594316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.594931 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.596857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.603775 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63099a6-f184-4c04-8b7d-13630c807afc admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.609270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.609870 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.610145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.610411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.611103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.613181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.626878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1261/2522] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:47.631868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.636791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1262/2523] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:47.641341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.654615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6bc4b-c547-456f-addd-a989122beee4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.698387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:47.704826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad4b7-d988-4ce4-bd02-2f26427678fc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.709755 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.710456 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.710721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.711016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.711634 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.713623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.725686 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.726310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.726623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.726899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.727688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.729332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.735907 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1262/2524] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:47.740539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1263/2525] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:47.740729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.744649 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.752547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621c695-c70b-4e10-b758-2ab0a429fcac admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.756300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ad1f-7ba9-4a37-b073-56a91770e72b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.758316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.759332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.759807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.760250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.761246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.761756 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.763170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.763536 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.763802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.763922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] PATH_INFO: `/v3/projects/91f743759fa748a0ba1911978d4419eb/users/cc82ade3fc544929a7040daca6475078/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.780668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1263/2526] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:47.780905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f743759fa748a0ba1911978d4419eb, user_id=cc82ade3fc544929a7040daca6475078, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.784084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.785385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.799218 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1264/2527] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:47 2022] PUT /identity/v3/projects/91f743759fa748a0ba1911978d4419eb/users/cc82ade3fc544929a7040daca6475078/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:47.799980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08511a72-3c01-4ef1-8202-fe420b643fca admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.803270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.869430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.870097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.870518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.870621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e7f59-0bad-4819-b24b-327c6b12fee4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.870814 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.871988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.873690 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.876131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.877049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.877327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.877459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:47.877623 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.878253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.880006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.891037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1264/2528] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:47.894167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.895286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1265/2529] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:47.896294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:47.904984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2bb4-676c-4bde-95b8-b923c1f265eb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.906900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21764b-aecb-49fc-8ee3-b0e21bebba87 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:47.909873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.910531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.910812 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.911118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] PATH_INFO: `/v3/projects/ea9e5e5acd7841c5920090a69b18937e/users/bc8aa62439234677a180f9a1d8a94f55/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.911597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:47.912261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.912528 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.912800 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] PATH_INFO: `/v3/projects/91f743759fa748a0ba1911978d4419eb/users/cc82ade3fc544929a7040daca6475078/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.915926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f743759fa748a0ba1911978d4419eb, user_id=cc82ade3fc544929a7040daca6475078, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.918648 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0d504-834c-43d4-810f-c56bf18960db admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.929421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9e5e5acd7841c5920090a69b18937e, user_id=bc8aa62439234677a180f9a1d8a94f55, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:47.930264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1266/2530] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:47 2022] PUT /identity/v3/projects/91f743759fa748a0ba1911978d4419eb/users/cc82ade3fc544929a7040daca6475078/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:47.932778 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e3bb-ad0b-43c5-9bad-daab10bd570c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:47.942290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.942579 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.942856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.948737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1265/2531] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:47 2022] PUT /identity/v3/projects/ea9e5e5acd7841c5920090a69b18937e/users/bc8aa62439234677a180f9a1d8a94f55/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:47.961592 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:47.961957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:47.962237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:47.979125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] MFA Rules not processed for user `cc82ade3fc544929a7040daca6475078`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:47.980346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:47.998197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] MFA Rules not processed for user `bc8aa62439234677a180f9a1d8a94f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:47.999464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.069914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1267/2532] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:48.077827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1266/2533] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:48.082000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:48.082000 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.089119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:48.091225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.112905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:48.152256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c45b1-35e3-4d85-8802-6f92a650cd9f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:48.156730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:48.157344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.157730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.157877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.169907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:48.171451 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:48.182851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaac21f-db2a-4bae-b969-30a1711ef25e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:48.185628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1268/2534] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:48.191489 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:48.192028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.192367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.192611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.202666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:48.204371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:48.218216 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1267/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:48.253059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:48.253059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.272568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bc42f-81f2-4aa6-9aa2-91dc32ec18ad service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:48.279540 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:48.280914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.287522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.288036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.380833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:48.382499 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd918ad0-9311-4a95-b072-dfc909a44948 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:48.404440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1269/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:48.490166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:48.550995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:48.568639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250e2a-12de-440c-b5e3-26d2fe327fef service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:48.574740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:48.575406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.575688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.575979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] PATH_INFO: `/v3/users/bc8aa62439234677a180f9a1d8a94f55` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.577130 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8aa62439234677a180f9a1d8a94f55)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:48.579242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:48.633768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8aa62439234677a180f9a1d8a94f55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:48.642918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8aa62439234677a180f9a1d8a94f55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:48.643758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8aa62439234677a180f9a1d8a94f55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:48.659293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8aa62439234677a180f9a1d8a94f55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:48.667226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8aa62439234677a180f9a1d8a94f55'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:48.673575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1268/2537] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:48 2022] DELETE /identity/v3/users/bc8aa62439234677a180f9a1d8a94f55 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:48.678081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.678331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.678587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.702818 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:48.704133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.772891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1270/2538] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:48.784582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:48.786052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:48.842724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44e17b-954e-4dac-9351-189a42465f9b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:48.846394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:48.847014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:48.847312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:48.847578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:48.857581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:48.859053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd4a98-bed1-4340-97eb-33ef54265b23 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:48.869577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1269/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:49.153545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.155092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:49.169971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9be3c7-338f-4ea4-81da-1cb6ad74f303 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.174653 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.175342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.175627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.175896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] PATH_INFO: `/v3/projects/ea9e5e5acd7841c5920090a69b18937e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.177184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9e5e5acd7841c5920090a69b18937e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.179154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.229694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9e5e5acd7841c5920090a69b18937e', 'request_id': 'req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:49.235367 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9e5e5acd7841c5920090a69b18937e', 'request_id': 'req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:49.236542 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:49.243652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9e5e5acd7841c5920090a69b18937e', 'request_id': 'req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:49.256681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] The token cache is being invalidate because project ea9e5e5acd7841c5920090a69b18937e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:49.257042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:49.257875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-6f5eaab5-e50d-4fc8-b077-bbf939c8993c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:49.259747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1271/2540] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:49 2022] DELETE /identity/v3/projects/ea9e5e5acd7841c5920090a69b18937e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:49.266966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.267322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.267560 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.287563 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:49.288772 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:49.348920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1270/2541] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:49.357009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.357293 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.357562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.378832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:49.380081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:49.388997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1272/2542] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:49.393346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.403987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5434b-8c0c-4a47-925f-c7b8675780ae None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.408882 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.409501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.409775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.410053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.410705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:49.411192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.412822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.420292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:49.426777 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1271/2543] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:49.431344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.443048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9723-3c3e-4ded-9546-686d99d23aa4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.448566 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.449279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.449589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.449856 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.450526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.452466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.488750 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1273/2544] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:49 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:49.493255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.505015 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96f6ba-7d84-4d7e-88ac-77bbe085d463 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.593343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.594210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.594519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.594854 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.595660 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.597986 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.639091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1272/2545] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:49 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:49.642896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.666322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ce87-9290-47aa-8785-a37feb8861f1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.673022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.673670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.674019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.674360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.675154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.676947 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.695964 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1274/2546] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:49.700542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.717034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea08ff-72b1-4363-b911-1e88f35af092 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.722302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.723227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.729897 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.730321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.731955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.733975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de09532-0976-46b6-8695-28ae68869a6c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.755446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1273/2547] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:49.759887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:49.780154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08100124-da54-4b2d-af48-0a54690ff892 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:49.815008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:49.816664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.816664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.816664 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] PATH_INFO: `/v3/projects/495654001a10428caed2ab86a72cceb1/users/659e771217a04867a468757a186196ab/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.845390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=495654001a10428caed2ab86a72cceb1, user_id=659e771217a04867a468757a186196ab, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:49.849682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:49.883483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1275/2548] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:49 2022] PUT /identity/v3/projects/495654001a10428caed2ab86a72cceb1/users/659e771217a04867a468757a186196ab/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:49.887025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:49.887296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:49.887593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:49.926605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] MFA Rules not processed for user `659e771217a04867a468757a186196ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:49.928096 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:50.035877 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1274/2549] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:50.040755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.206299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a57314-4c59-4fed-bc40-a620122bfddf admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.211877 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.212602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.213732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.214115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.215042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.217371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.268565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1276/2550] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:50 2022] POST /identity/v3/projects => generated 371 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:50.274275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.286918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06d70d-31a5-42f1-8c83-08a3c931d3a1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.369245 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.370948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.371276 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.371643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.372121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.375512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.389645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:50.419122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1275/2551] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:50 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:50.420240 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.442890 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd70d10-d912-4ea5-8c42-6c5e15a0245d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.448963 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.449782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.450095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.450527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.451480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.453325 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.467617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1277/2552] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:50.472230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.483292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159d323-6721-4a67-acf5-281e849cfc67 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.488285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.489024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.489300 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.489574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] PATH_INFO: `/v3/projects/d0f4ea3a74764d23ae197860f82d6708/users/b0ff697e88cd4ac5b45b37e8c937c8bc/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.507056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f4ea3a74764d23ae197860f82d6708, user_id=b0ff697e88cd4ac5b45b37e8c937c8bc, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.517522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.552280 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1276/2553] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:50 2022] PUT /identity/v3/projects/d0f4ea3a74764d23ae197860f82d6708/users/b0ff697e88cd4ac5b45b37e8c937c8bc/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:50.556406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.570270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ecc53-c812-4a57-99d8-763e7a91121a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.704075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:50.713822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.714773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.715304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.715639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.716370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.718126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83fa77-a351-4079-81a5-9cf06741be7c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.738357 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1278/2554] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:50.739799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.753621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b300c9-5226-45e6-9a49-afcbb2133c4c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:50.783590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:50.784094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.784369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.784643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] PATH_INFO: `/v3/projects/d0f4ea3a74764d23ae197860f82d6708/users/b0ff697e88cd4ac5b45b37e8c937c8bc/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.786811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f4ea3a74764d23ae197860f82d6708, user_id=b0ff697e88cd4ac5b45b37e8c937c8bc, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:50.789762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:50.805439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1277/2555] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:50 2022] PUT /identity/v3/projects/d0f4ea3a74764d23ae197860f82d6708/users/b0ff697e88cd4ac5b45b37e8c937c8bc/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:50.820736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:50.827389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:50.827944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:50.873092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] MFA Rules not processed for user `b0ff697e88cd4ac5b45b37e8c937c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:50.874412 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:50.976327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1279/2556] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:50.984108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:50.985833 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:51.056293 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d9f66-15b2-4d2b-a67d-272b9d3c529f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:51.063122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:51.063759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:51.064034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:51.064305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:51.076462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:51.078601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b917f-35ba-4091-9d84-c13149d3d514 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:51.093432 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1278/2557] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:51.665842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:51.755572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac66f6-07b1-4540-bff3-c48711369620 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:51.763302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:51.764389 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:51.765006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:51.765555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] PATH_INFO: `/v3/users/659e771217a04867a468757a186196ab` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:51.769588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659e771217a04867a468757a186196ab)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:51.770760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:51.831790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659e771217a04867a468757a186196ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:51.839874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659e771217a04867a468757a186196ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:51.841349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659e771217a04867a468757a186196ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:51.852223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659e771217a04867a468757a186196ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:51.878885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659e771217a04867a468757a186196ab'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:51.880699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:51.889982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1280/2558] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:51 2022] DELETE /identity/v3/users/659e771217a04867a468757a186196ab => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:51.894555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:51.895149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:51.895149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:51.924881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:51.926313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:51.997100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1279/2559] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:52.009500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:52.011275 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.100256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65263c-b864-475e-8dfb-a851bbf5b4e2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:52.103889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:52.104512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:52.104789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:52.105059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:52.121553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:52.123029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f6bc7-e7ba-44d9-9f25-588948580217 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:52.140232 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1281/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:52.156464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:52.158500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.173157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1972-d0eb-4b4a-a063-44206677df22 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:52.178004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:52.178666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:52.179156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:52.179618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] PATH_INFO: `/v3/users/076f374c1f8041d4b5eadaa2bd9870ff` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:52.180613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] RBAC: Authorizing `identity:delete_user(user_id=076f374c1f8041d4b5eadaa2bd9870ff)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:52.182715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:52.256150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076f374c1f8041d4b5eadaa2bd9870ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.265290 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076f374c1f8041d4b5eadaa2bd9870ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.265981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076f374c1f8041d4b5eadaa2bd9870ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.268322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:52.288557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076f374c1f8041d4b5eadaa2bd9870ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.312947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076f374c1f8041d4b5eadaa2bd9870ff'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.335788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1280/2561] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:52 2022] DELETE /identity/v3/users/076f374c1f8041d4b5eadaa2bd9870ff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:52.336894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:52.337176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:52.337565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:52.388848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:52.388848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.484219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1282/2562] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:52.499185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:52.499185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.576581 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb1775-55a1-4cf3-82a4-69e962b8024d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:52.580256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:52.580866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:52.581145 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:52.581416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:52.591378 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:52.592734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:52.602672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1281/2563] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:52.854792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:52.857448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.871415 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e0c6d-1181-4720-a23c-4398cba55120 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:52.877568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:52.878734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:52.878734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:52.878951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] PATH_INFO: `/v3/projects/495654001a10428caed2ab86a72cceb1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:52.880259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=495654001a10428caed2ab86a72cceb1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:52.882335 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:52.941037 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:52.942631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:52.951752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495654001a10428caed2ab86a72cceb1', 'request_id': 'req-84099822-0fcf-4b7f-89c6-b0406a4938ed'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.960668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495654001a10428caed2ab86a72cceb1', 'request_id': 'req-84099822-0fcf-4b7f-89c6-b0406a4938ed'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:52.964284 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268bbef-1fc4-4761-b7e3-dc7d43ce917f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:52.971217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495654001a10428caed2ab86a72cceb1', 'request_id': 'req-84099822-0fcf-4b7f-89c6-b0406a4938ed'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.003290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] The token cache is being invalidate because project 495654001a10428caed2ab86a72cceb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:53.003680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.004309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.009796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1283/2564] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:52 2022] DELETE /identity/v3/projects/495654001a10428caed2ab86a72cceb1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:53.010433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:53.011896 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.045031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:53.075898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:53.076552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.077025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.077302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] PATH_INFO: `/v3/projects/13321ac853bb4b709a88e2eba6e88d79` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.078577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13321ac853bb4b709a88e2eba6e88d79)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:53.081008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:53.113337 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84099822-0fcf-4b7f-89c6-b0406a4938ed admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:53.119773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:53.120732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.120861 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.121076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] PATH_INFO: `/v3/users/b0ff697e88cd4ac5b45b37e8c937c8bc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.123119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ff697e88cd4ac5b45b37e8c937c8bc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:53.125115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:53.171076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13321ac853bb4b709a88e2eba6e88d79', 'request_id': 'req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.181288 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13321ac853bb4b709a88e2eba6e88d79', 'request_id': 'req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.190069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13321ac853bb4b709a88e2eba6e88d79', 'request_id': 'req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.200149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ff697e88cd4ac5b45b37e8c937c8bc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.204217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] The token cache is being invalidate because project 13321ac853bb4b709a88e2eba6e88d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:53.204498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.205875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.214506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1282/2565] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:52 2022] DELETE /identity/v3/projects/13321ac853bb4b709a88e2eba6e88d79 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:53.219428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:53.222097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.239595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ff697e88cd4ac5b45b37e8c937c8bc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.241463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ff697e88cd4ac5b45b37e8c937c8bc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.244562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:53.262702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ff697e88cd4ac5b45b37e8c937c8bc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.281780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ff697e88cd4ac5b45b37e8c937c8bc'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.287983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1284/2566] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:53 2022] DELETE /identity/v3/users/b0ff697e88cd4ac5b45b37e8c937c8bc => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:53.321335 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a3f1-4a96-4a95-af5b-0eaef5fcf918 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:53.392753 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:53.393299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.393576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.393849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] PATH_INFO: `/v3/users/cc82ade3fc544929a7040daca6475078` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.395031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc82ade3fc544929a7040daca6475078)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:53.397064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:53.455615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc82ade3fc544929a7040daca6475078'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.484721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc82ade3fc544929a7040daca6475078'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.485368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc82ade3fc544929a7040daca6475078'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.499962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc82ade3fc544929a7040daca6475078'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.508218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc82ade3fc544929a7040daca6475078'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.515062 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1283/2567] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:53 2022] DELETE /identity/v3/users/cc82ade3fc544929a7040daca6475078 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:53.630472 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:53.633706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.666424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32da98-69ad-4b57-bc89-cbc98c62310b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:53.738813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:53.739486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.739768 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.740042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] PATH_INFO: `/v3/projects/d0f4ea3a74764d23ae197860f82d6708` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.741175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f4ea3a74764d23ae197860f82d6708)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:53.743153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:53.809929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:53.814798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f4ea3a74764d23ae197860f82d6708', 'request_id': 'req-c0174f8e-6b9a-401b-af1b-0612349a876b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.823354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f4ea3a74764d23ae197860f82d6708', 'request_id': 'req-c0174f8e-6b9a-401b-af1b-0612349a876b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.831055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f4ea3a74764d23ae197860f82d6708', 'request_id': 'req-c0174f8e-6b9a-401b-af1b-0612349a876b'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.837718 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:53.839479 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.847083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] The token cache is being invalidate because project d0f4ea3a74764d23ae197860f82d6708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:53.847380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.848035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-c0174f8e-6b9a-401b-af1b-0612349a876b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:53.851012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1285/2568] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:53 2022] DELETE /identity/v3/projects/d0f4ea3a74764d23ae197860f82d6708 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:53.854943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57dfe-5a6e-45f7-b013-1a04685d9b91 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.855657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfaeaac-c509-4f0a-90f9-d7afcdb0a978 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:53.856296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57dfe-5a6e-45f7-b013-1a04685d9b91 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.856296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57dfe-5a6e-45f7-b013-1a04685d9b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.883830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7cb57dfe-5a6e-45f7-b013-1a04685d9b91 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:53.885370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7cb57dfe-5a6e-45f7-b013-1a04685d9b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.902962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:53.938730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:53.939393 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.939675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.939946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] PATH_INFO: `/v3/projects/91f743759fa748a0ba1911978d4419eb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.941237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f743759fa748a0ba1911978d4419eb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:53.943144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:53.952955 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1286/2569] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:53.961670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:53.962025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:53.962553 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:53.985519 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:53.986825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:53.997212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f743759fa748a0ba1911978d4419eb', 'request_id': 'req-150ecd46-71e7-45c1-9cb4-7a1e21499972'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:54.005141 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f743759fa748a0ba1911978d4419eb', 'request_id': 'req-150ecd46-71e7-45c1-9cb4-7a1e21499972'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:54.013641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f743759fa748a0ba1911978d4419eb', 'request_id': 'req-150ecd46-71e7-45c1-9cb4-7a1e21499972'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:54.025616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] The token cache is being invalidate because project 91f743759fa748a0ba1911978d4419eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:54.025926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:54.026582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-150ecd46-71e7-45c1-9cb4-7a1e21499972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:54.028669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1284/2570] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:53 2022] DELETE /identity/v3/projects/91f743759fa748a0ba1911978d4419eb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:54.033558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5ba9-0f40-4a2f-8786-87590d59139c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.033838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5ba9-0f40-4a2f-8786-87590d59139c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.034110 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f5ba9-0f40-4a2f-8786-87590d59139c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.053984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1287/2571] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:54.058230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.058783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1a2f5ba9-0f40-4a2f-8786-87590d59139c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:54.060034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1a2f5ba9-0f40-4a2f-8786-87590d59139c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:54.068435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d0b3c-2c1a-4ab9-9720-f4b93acbd4e0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.071548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1285/2572] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:54.075005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.075639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.075935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.076191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.076851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:54.077313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.079069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.081918 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.082416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.082729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.097269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1288/2573] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:03:54.098699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.103052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:54.104266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:54.111937 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e24f1-c33f-47f1-b68a-fec5fe88b847 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.112956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1286/2574] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:54.118284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.118589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.118991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.119291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.119562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.120191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.125131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.129878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ed15-4028-4e62-9423-21791ffac413 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.137540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.138172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.138451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.138734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.139497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:54.139801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.141413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.154428 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1287/2575] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:03:54.158711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1289/2576] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/projects => generated 401 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:54.158856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.162656 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.169953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c0ee5-174f-4e87-8131-b3b89d5d3570 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.209296 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:54.237264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29d17-3fa5-4aaa-93e9-0bc396322419 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.242809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.243602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.243881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.244188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.244787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.246785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.249369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.249994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.250272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.250540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.251218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.253490 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.278248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1290/2577] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:54.279682 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.292348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94b50-402c-4341-8059-dfab58538ae0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.297918 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.298593 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.298905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.299356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.300007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.303101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.309645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1288/2578] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:54.314783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.323803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1291/2579] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:54.329792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.340941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb933722-f69f-48d4-b19c-9dd4368bd9d8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.356340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:54.408281 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf065c-7058-4fab-be15-fea1a3b190d4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.414757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.415586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.415979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.416346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.417549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.420822 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.456701 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1289/2580] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:54.460980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.472053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd396962-f68e-49aa-8966-a3c3a8736aa3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.478631 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.484407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.484407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.484407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.484407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.484407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.499156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1290/2581] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:54.505348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.519860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e226f-a9d7-4674-bb96-b8f275c44fd5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.526365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.527075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.527541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.527821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.528476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.530392 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.545374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1291/2582] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:54.546462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.557088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd45e6-a00a-4e9a-80f6-0dd6114f7e1b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.562370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.563140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.563491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.563884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] PATH_INFO: `/v3/projects/968e0e821f6745009963cc7a325a9735/users/d73d1ba2e2f149c2ace0c9caa87781b8/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.575224 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.575844 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.576132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.576574 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.577289 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.578829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.583464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=968e0e821f6745009963cc7a325a9735, user_id=d73d1ba2e2f149c2ace0c9caa87781b8, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.586693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82d2da-486c-4579-bd71-474c43fe98e5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.589645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1292/2583] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:54.594436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.609026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453784a3-67c5-4128-860e-7181a7626ae3 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.613192 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1292/2584] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:54 2022] PUT /identity/v3/projects/968e0e821f6745009963cc7a325a9735/users/d73d1ba2e2f149c2ace0c9caa87781b8/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:54.624439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.624733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.625157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.657028 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] MFA Rules not processed for user `d73d1ba2e2f149c2ace0c9caa87781b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:54.658354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:54.671498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.672546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.672826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.673094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] PATH_INFO: `/v3/projects/da07160c9b094e12b18b78049f608c7f/users/f902f70b1ebb48f887faa4a00a7896cd/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.689929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da07160c9b094e12b18b78049f608c7f, user_id=f902f70b1ebb48f887faa4a00a7896cd, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.692637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9ab3a-d614-46cf-bc6c-66853faa503c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.708498 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1293/2585] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:54 2022] PUT /identity/v3/projects/da07160c9b094e12b18b78049f608c7f/users/f902f70b1ebb48f887faa4a00a7896cd/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:54.727239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.727779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.728050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.749124 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1293/2586] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:54.754885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.758468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] MFA Rules not processed for user `f902f70b1ebb48f887faa4a00a7896cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:54.759677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:54.767861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb2aa1-f18e-4123-8200-966385e49102 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.794463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:54.829658 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:54.839343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.841073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.841668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.842059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.842661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1294/2587] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:54.843113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.845083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.846853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.858336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988a12c-01b0-4b1d-9c10-c18b65def32a None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.863420 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.864110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.864378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.864692 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.865482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.867680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:54.874134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1294/2588] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:54.878237 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.903954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1295/2589] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:54.907929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:54.950665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837635b-63b6-4d5d-b4f1-89aa6b88da1b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.989608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9aee7-00d0-473e-94d9-b7a0b7d64882 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:54.994285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:54.994882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:54.995186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:54.995454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:54.996167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:54.999096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.017389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.018107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.018394 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.018673 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.019340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.021883 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.028174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1296/2590] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:55.032787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.044729 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac036e-6f20-4235-9480-72f9d3f735ae admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.045370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1295/2591] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:54 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:03:55.049185 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.051610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.052287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.052637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.052984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.053699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.055449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.063535 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098997f3-b18e-4e45-85fc-118d9e02f7ef admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.068326 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.068930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.069206 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.069485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.070125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.071750 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.075130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1297/2592] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:55.078157 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.088471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1296/2593] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:55.088699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945e5ce-0bcd-4f6a-a097-4813ef4bdf09 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.093254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.094401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.095105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.095399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.095707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] PATH_INFO: `/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4/users/5ccb5841f6634ca6a36fca26b966b853/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.109322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab566-1082-4400-9050-240549e1a2e0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.114396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.115139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.115329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.115610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] PATH_INFO: `/v3/projects/15bf83ce9c064c1c9cd65359389f4318/users/bf938354c3cb423b9e7ff88da375798a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.115919 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc8354c3ddd4ec79e6d20c0a717d1a4, user_id=5ccb5841f6634ca6a36fca26b966b853, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.118602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.132664 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bf83ce9c064c1c9cd65359389f4318, user_id=bf938354c3cb423b9e7ff88da375798a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.133430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1298/2594] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:55 2022] PUT /identity/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4/users/5ccb5841f6634ca6a36fca26b966b853/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:55.135386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.137904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.151611 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1297/2595] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:55 2022] PUT /identity/v3/projects/15bf83ce9c064c1c9cd65359389f4318/users/bf938354c3cb423b9e7ff88da375798a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:55.155860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.206921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:55.207265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa6d00-b944-4e05-a59b-04a8ba331670 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.211592 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:55.225696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3d130-d425-4905-8bfa-8004bdd1ac14 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.230451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.231106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.231388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.231754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.232401 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.234012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.247034 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1298/2596] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:55.249967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.264956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3042d5-06d1-449c-afd3-e0e6be72dbbc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.274073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.274868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.275176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.275488 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] PATH_INFO: `/v3/projects/15bf83ce9c064c1c9cd65359389f4318/users/bf938354c3cb423b9e7ff88da375798a/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.277577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15bf83ce9c064c1c9cd65359389f4318, user_id=bf938354c3cb423b9e7ff88da375798a, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.279302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.279917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.280195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.280482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.280597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.281134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.283131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750b2f5-61cd-4f95-96b3-5e94025509c2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.293345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1299/2597] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:55 2022] PUT /identity/v3/projects/15bf83ce9c064c1c9cd65359389f4318/users/bf938354c3cb423b9e7ff88da375798a/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:55.297245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1299/2598] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:55.301353 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.304490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.304781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.305148 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.339588 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] MFA Rules not processed for user `bf938354c3cb423b9e7ff88da375798a`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:55.341445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.374674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bda0a-dd62-4ee7-8264-59e5899bb129 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.380651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.381304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.381583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.381859 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] PATH_INFO: `/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4/users/5ccb5841f6634ca6a36fca26b966b853/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.383855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc8354c3ddd4ec79e6d20c0a717d1a4, user_id=5ccb5841f6634ca6a36fca26b966b853, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.386539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f5837-9d8e-45c5-ba34-17e77fd1aa1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.398989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1300/2599] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:55 2022] PUT /identity/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4/users/5ccb5841f6634ca6a36fca26b966b853/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:55.409996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.410282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.410554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.422082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1300/2600] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:55.434088 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.435475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.440512 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] MFA Rules not processed for user `5ccb5841f6634ca6a36fca26b966b853`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:55.441730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.503559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2f1e8a-480d-404f-8e39-1c4e94ff36de None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.537715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1301/2601] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:55.550216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:55.550574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.552224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.562095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.562811 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.564855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.564855 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.609527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:55.622638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f64c7-3b23-48a5-a39a-cc064984d678 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.626347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.627315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.627791 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.628067 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.634248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.635709 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.637912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.639361 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.647208 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1301/2602] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:55.649646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1302/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:55.754312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.795458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.815688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545659-f5ba-4574-abf2-de35375694d5 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.819579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.820196 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.820453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.820721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.842774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06982819-09df-4a67-a564-2cf48414bd59 service cinder] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.847641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:55.848294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.848548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.849638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] PATH_INFO: `/v3/users/d73d1ba2e2f149c2ace0c9caa87781b8` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.850021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73d1ba2e2f149c2ace0c9caa87781b8)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.852304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.890534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:55.892184 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:55.903132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1303/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:03:55.903871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:55.905863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.909992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73d1ba2e2f149c2ace0c9caa87781b8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:55.922414 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73d1ba2e2f149c2ace0c9caa87781b8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:55.923748 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73d1ba2e2f149c2ace0c9caa87781b8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:55.933131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73d1ba2e2f149c2ace0c9caa87781b8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:55.949126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-4278973b-eda5-4c9a-b5e0-c390bb067b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73d1ba2e2f149c2ace0c9caa87781b8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:55.958386 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1302/2605] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:55 2022] DELETE /identity/v3/users/d73d1ba2e2f149c2ace0c9caa87781b8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:55.964617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:55.965166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:55.965610 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:55.986373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e98759-36ee-4084-9fda-31c09df87445 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:55.986846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:55.988105 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:55.998555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:56.047313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:56.047927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:56.048203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:56.048477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:56.050298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:56.079373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1303/2606] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:56.099246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:56.101605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:56.119570 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614da09-0bd5-440f-8c8a-a2625aef3d8f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:56.122794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:56.125879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:56.126987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:56.127404 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:56.127754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:56.138100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:56.139465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:56.139789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:56.141949 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:56.154694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1304/2607] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:56.164661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1304/2608] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:56.690298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:56.692080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:56.706829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83156f6d-9b6c-4303-8de3-4a8a4b2724e5 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:56.712233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:56.712940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:56.713417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:56.713758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] PATH_INFO: `/v3/projects/968e0e821f6745009963cc7a325a9735` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:56.715164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968e0e821f6745009963cc7a325a9735)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:56.717469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:56.779403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968e0e821f6745009963cc7a325a9735', 'request_id': 'req-bfc9bdb8-57fe-4c11-a9f9-537e01394562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.787853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968e0e821f6745009963cc7a325a9735', 'request_id': 'req-bfc9bdb8-57fe-4c11-a9f9-537e01394562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.795447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968e0e821f6745009963cc7a325a9735', 'request_id': 'req-bfc9bdb8-57fe-4c11-a9f9-537e01394562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.819599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] The token cache is being invalidate because project 968e0e821f6745009963cc7a325a9735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:56.819599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.819599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.821895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1305/2609] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:56 2022] DELETE /identity/v3/projects/968e0e821f6745009963cc7a325a9735 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:56.825844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:56.836367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823dbb4-6f6e-4a04-9fee-6cd46de1ef54 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:56.899294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:56.899945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:56.900223 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:56.900495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] PATH_INFO: `/v3/users/bf938354c3cb423b9e7ff88da375798a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:56.901681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf938354c3cb423b9e7ff88da375798a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:56.904249 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:56.958356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf938354c3cb423b9e7ff88da375798a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.968251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf938354c3cb423b9e7ff88da375798a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.968951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf938354c3cb423b9e7ff88da375798a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.983948 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf938354c3cb423b9e7ff88da375798a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:56.991670 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8663e48d-1849-422f-9797-c9c1b16ef555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf938354c3cb423b9e7ff88da375798a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.001926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1305/2610] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:56 2022] DELETE /identity/v3/users/bf938354c3cb423b9e7ff88da375798a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:57.307229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:57.315700 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:57.330430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9bdb8-57fe-4c11-a9f9-537e01394562 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:57.404931 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:57.405564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.405843 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.406340 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] PATH_INFO: `/v3/projects/15bf83ce9c064c1c9cd65359389f4318` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.407301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15bf83ce9c064c1c9cd65359389f4318)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:57.409396 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:57.461305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:57.486550 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15bf83ce9c064c1c9cd65359389f4318', 'request_id': 'req-42b76374-7431-462d-99d1-fb967ddb3160'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.492829 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15bf83ce9c064c1c9cd65359389f4318', 'request_id': 'req-42b76374-7431-462d-99d1-fb967ddb3160'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.500815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15bf83ce9c064c1c9cd65359389f4318', 'request_id': 'req-42b76374-7431-462d-99d1-fb967ddb3160'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.516765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] The token cache is being invalidate because project 15bf83ce9c064c1c9cd65359389f4318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:03:57.517666 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.518526 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-42b76374-7431-462d-99d1-fb967ddb3160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:57.523663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1306/2611] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:03:57 2022] DELETE /identity/v3/projects/15bf83ce9c064c1c9cd65359389f4318 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:57.531348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.531688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.531953 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.552653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:57.554045 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:57.657661 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1306/2612] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:57.660047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.660533 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.660914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.724277 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:57.731513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:57.741968 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1307/2613] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:57.748421 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:57.759906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3fc9c-136d-414b-a15d-5edb3e0dc51b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:57.765154 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:57.765851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.766352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.766742 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.767448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:03:57.767813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:57.769447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:57.780610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:57.786759 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1307/2614] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:03:57.791719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:57.805005 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b9cf-8d9f-4a9f-a9cd-4063570562ea None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:57.810716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:57.811353 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.811629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.811901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.812530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:57.814426 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:57.852862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1308/2615] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:57 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:57.853765 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:57.864525 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edea524-f833-45d0-959e-067f3e055e9d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:57.929116 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:57.929832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.930150 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.930520 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.931250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:57.933343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:57.967993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1308/2616] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:57 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:57.969174 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:57.979755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d29a3-b596-48ca-ab35-40d2f6aabc38 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:57.985535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:57.986347 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:57.986624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:57.986893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:57.987573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:57.989757 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.005028 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1309/2617] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:58.008590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.020031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47d84c-bccc-4a2f-b63e-0ececf821934 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.025646 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.026323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.026632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.026912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.030023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.030023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809c58b-73df-441b-9575-16a5164c702c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.041057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1309/2618] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:58.046561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.056979 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fbb07-8aef-41a6-956c-5874f065cc5d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.064125 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.064780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.065607 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.065746 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] PATH_INFO: `/v3/projects/e9b6267762874c5ea78f91635f6a85cb/users/58e974bf61ce4eb8914cbbf413b6f34d/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.081713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b6267762874c5ea78f91635f6a85cb, user_id=58e974bf61ce4eb8914cbbf413b6f34d, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.084808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.097710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1310/2619] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:58 2022] PUT /identity/v3/projects/e9b6267762874c5ea78f91635f6a85cb/users/58e974bf61ce4eb8914cbbf413b6f34d/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:58.108727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.109127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.109473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.139541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] MFA Rules not processed for user `58e974bf61ce4eb8914cbbf413b6f34d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:58.141104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:58.255417 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1310/2620] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:58.260504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.342016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6a037-2b33-45cb-8f59-bdb530a2348a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.346782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.347480 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.347916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.348307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.349036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.351178 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.387453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1311/2621] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:03:58 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:58.391738 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.408455 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264becc1-4526-4043-9c0d-2d229d3ba7df None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.481538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.482140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.482470 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.482746 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.483675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.487128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.509047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:58.526085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1311/2622] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:03:58 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:03:58.530264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.540891 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c0cad-65cf-4290-b564-5865a456fbd4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.547043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.548042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.548425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.548796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.549793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.551327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.572736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1312/2623] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:03:58.574040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.584894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91948f8-b173-44bc-beef-6952592820ec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.590726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.591458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.591786 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.592405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] PATH_INFO: `/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc/users/71483c0faae14b71bcbfa7b57bb805f1/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.609696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0d43a9e0b24bc4b453c6d767970cbc, user_id=71483c0faae14b71bcbfa7b57bb805f1, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.612641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.630774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1312/2624] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:58 2022] PUT /identity/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc/users/71483c0faae14b71bcbfa7b57bb805f1/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:58.633256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.643557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a0823-272f-4f0b-83db-996243f02bfa admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.666823 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:58.721139 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.721742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.722019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.722288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.722929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.724583 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02479c40-7747-4774-b76a-e3b0c8369719 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.737764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1313/2625] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:03:58.742075 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.757285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9304b-6417-431d-a8cd-8933b40c7b43 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:58.763106 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:58.763771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.764225 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.764569 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] PATH_INFO: `/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc/users/71483c0faae14b71bcbfa7b57bb805f1/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.768017 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0d43a9e0b24bc4b453c6d767970cbc, user_id=71483c0faae14b71bcbfa7b57bb805f1, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:58.771578 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:58.795497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1313/2626] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:03:58 2022] PUT /identity/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc/users/71483c0faae14b71bcbfa7b57bb805f1/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:03:58.806392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:58.806567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:58.806816 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:58.848596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] MFA Rules not processed for user `71483c0faae14b71bcbfa7b57bb805f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:58.850531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:58.945808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1314/2627] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:03:58.963622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:58.965464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:59.043834 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cfe27-04bb-4c36-bde8-34c30b605b19 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:59.047422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:59.048032 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:59.048302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:59.048568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:59.063795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:59.065946 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2998f4-0cee-4197-83a4-3d7da06df2ca service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:59.081485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1314/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:03:59.534810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:59.658715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476ada1-d9f0-4d9f-bbfe-7cf98dd1a8cf None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:03:59.663871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:03:59.664747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:59.665310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:59.665742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] PATH_INFO: `/v3/users/58e974bf61ce4eb8914cbbf413b6f34d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:59.667216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58e974bf61ce4eb8914cbbf413b6f34d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:03:59.669906 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:03:59.745297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58e974bf61ce4eb8914cbbf413b6f34d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:59.748027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:03:59.760981 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58e974bf61ce4eb8914cbbf413b6f34d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:59.762362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58e974bf61ce4eb8914cbbf413b6f34d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:59.771743 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58e974bf61ce4eb8914cbbf413b6f34d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:59.807346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58e974bf61ce4eb8914cbbf413b6f34d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:03:59.815581 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1315/2629] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:03:59 2022] DELETE /identity/v3/users/58e974bf61ce4eb8914cbbf413b6f34d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:03:59.820515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:03:59.820973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:03:59.821347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:03:59.845339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:03:59.848231 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:03:59.963794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1315/2630] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:03:59.984075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:03:59.989882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.018768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.020221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.071828 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fec5a-9dbd-49f2-a873-8ac4aff1b7a4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.075887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.076513 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.076796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.077069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.087836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.090264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.101575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1316/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:00.102527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.104016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.119697 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049df6f-c653-4d7c-ad23-c2076ee33c4d service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.123801 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:00.125688 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.126833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.127093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.127661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] PATH_INFO: `/v3/users/18bf10b172e7450a9b618683643ed48e` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.133853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bf10b172e7450a9b618683643ed48e)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.139301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.140251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d408-8c9d-4a24-9a57-5ee9df034919 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.147975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.148600 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.148878 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.149643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.155721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.180911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.182527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.196426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1316/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:04:00.223424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bf10b172e7450a9b618683643ed48e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.241313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bf10b172e7450a9b618683643ed48e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.242859 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bf10b172e7450a9b618683643ed48e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.254075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bf10b172e7450a9b618683643ed48e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.276990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bf10b172e7450a9b618683643ed48e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.284796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1317/2633] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:00 2022] DELETE /identity/v3/users/18bf10b172e7450a9b618683643ed48e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:00.352978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.354762 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.451858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504b564-77fa-48dd-b547-862f77f78360 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.490019 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.493880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.534917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.540169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.540496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.540880 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] PATH_INFO: `/v3/users/f902f70b1ebb48f887faa4a00a7896cd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.543788 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f902f70b1ebb48f887faa4a00a7896cd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.546234 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.573705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:00.601125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f902f70b1ebb48f887faa4a00a7896cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.609808 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f902f70b1ebb48f887faa4a00a7896cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.610696 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f902f70b1ebb48f887faa4a00a7896cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.625354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f902f70b1ebb48f887faa4a00a7896cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.633985 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f902f70b1ebb48f887faa4a00a7896cd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.638628 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1bf83-4d71-4aae-8965-b1e4cc60d076 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.640481 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1317/2634] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:00 2022] DELETE /identity/v3/users/f902f70b1ebb48f887faa4a00a7896cd => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:00.641547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.642881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.654104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:00.700598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.701236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.702387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.702687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] PATH_INFO: `/v3/projects/e9b6267762874c5ea78f91635f6a85cb` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.703894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b6267762874c5ea78f91635f6a85cb)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.705929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.721682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69806280-7a7f-4c3b-ac93-e88ffe9c6231 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.726385 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.727036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.727315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.727587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] PATH_INFO: `/v3/projects/3b320e561ec5450682ed979963c392a0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.728709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b320e561ec5450682ed979963c392a0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.731132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.762611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b6267762874c5ea78f91635f6a85cb', 'request_id': 'req-63551b54-cd65-4200-aa4d-6a5b877d8f86'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.768124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b6267762874c5ea78f91635f6a85cb', 'request_id': 'req-63551b54-cd65-4200-aa4d-6a5b877d8f86'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.774258 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b6267762874c5ea78f91635f6a85cb', 'request_id': 'req-63551b54-cd65-4200-aa4d-6a5b877d8f86'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.783443 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b320e561ec5450682ed979963c392a0', 'request_id': 'req-94864494-16f2-405a-8fb6-deeee1a06fe6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.787986 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] The token cache is being invalidate because project e9b6267762874c5ea78f91635f6a85cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:00.788432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.789149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-63551b54-cd65-4200-aa4d-6a5b877d8f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.791060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1318/2635] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:00 2022] DELETE /identity/v3/projects/e9b6267762874c5ea78f91635f6a85cb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:00.791538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b320e561ec5450682ed979963c392a0', 'request_id': 'req-94864494-16f2-405a-8fb6-deeee1a06fe6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.792530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca647-31eb-4fbe-ad7f-6dee9d10c48c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.792836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca647-31eb-4fbe-ad7f-6dee9d10c48c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.793600 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ca647-31eb-4fbe-ad7f-6dee9d10c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.799794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b320e561ec5450682ed979963c392a0', 'request_id': 'req-94864494-16f2-405a-8fb6-deeee1a06fe6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.811510 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] The token cache is being invalidate because project 3b320e561ec5450682ed979963c392a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:00.811805 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.812484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.814024 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1318/2636] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:00 2022] DELETE /identity/v3/projects/3b320e561ec5450682ed979963c392a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:00.814861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.816233 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.826721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e70ca647-31eb-4fbe-ad7f-6dee9d10c48c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:00.828131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e70ca647-31eb-4fbe-ad7f-6dee9d10c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.865301 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:00.885914 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94864494-16f2-405a-8fb6-deeee1a06fe6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:00.890908 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:00.891562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.891839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.892114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] PATH_INFO: `/v3/users/71483c0faae14b71bcbfa7b57bb805f1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.893241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71483c0faae14b71bcbfa7b57bb805f1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:00.894050 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1319/2637] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:00.895305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.895550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:00.895740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.895985 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:00.947031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:00.951009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:00.952797 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.954679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71483c0faae14b71bcbfa7b57bb805f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.961365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1320/2638] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:00.962229 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:00.963624 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:00.964534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71483c0faae14b71bcbfa7b57bb805f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.965419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71483c0faae14b71bcbfa7b57bb805f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.978537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71483c0faae14b71bcbfa7b57bb805f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.985768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6dfe3be5-83c7-4cb0-8445-a8a726a50e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71483c0faae14b71bcbfa7b57bb805f1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:00.992268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1319/2639] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:00 2022] DELETE /identity/v3/users/71483c0faae14b71bcbfa7b57bb805f1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:00.995474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:00.995474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:00.995474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.016794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:01.018074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.096104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d68ba-7f7f-4d2c-9d40-54c9f9789efb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.132916 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1320/2640] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:01.138657 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.148654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151aac4e-63b6-4c71-9966-54d0eaeb5f88 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.155069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.155702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.156070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.156471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.157190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:01.157562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.159264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.161012 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.161580 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.161858 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.162128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.164465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.176472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1321/2641] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:04:01.177549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.181403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.183076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.193416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa6917-fb3b-4a42-87a2-1abccfc2f264 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.195821 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1321/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:01.202167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.202930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.203305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.203648 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.204422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.206819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.238960 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1322/2643] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:04:01.240428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.250145 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ed519-ee1c-4873-bcd1-b37a81954bdd service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.310354 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:01.321501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.322102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.322379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.322647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.323306 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.326316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.352368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1322/2644] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:01.356278 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.366959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69d92-50fb-4242-9357-76ad49a61214 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.371582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.372204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.372450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.372739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.373371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.374955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.385643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1323/2645] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:01.390314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.403434 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a65719-f96e-41c1-9e14-f7166f4e8f95 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.408764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.409668 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.409961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.410481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.411162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.412897 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.431363 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1323/2646] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:01.433244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.450213 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc6ae3-d193-4ae4-92f4-51cac0600de2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.457176 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.460378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.475826 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.480104 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.482077 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.483036 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] PATH_INFO: `/v3/projects/368b233309724d538d4756ec424f15df/users/56a850972bd44327b7a1328e65769b9e/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.490837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cca18-1b97-4ff5-8cc8-549d0c1ca790 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.501707 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.502926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.503401 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.503893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] PATH_INFO: `/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.505651 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0d43a9e0b24bc4b453c6d767970cbc)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.509327 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.529466 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] RBAC: Authorizing `identity:create_grant(project_id=368b233309724d538d4756ec424f15df, user_id=56a850972bd44327b7a1328e65769b9e, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.538086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f810df-15a0-49fa-b426-0b7ac9db958f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.563009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1324/2647] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:01 2022] PUT /identity/v3/projects/368b233309724d538d4756ec424f15df/users/56a850972bd44327b7a1328e65769b9e/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:01.574996 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ddd-aed9-4e76-847c-abc1ecf0a6cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.575460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ddd-aed9-4e76-847c-abc1ecf0a6cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.575967 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ddd-aed9-4e76-847c-abc1ecf0a6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.595554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0d43a9e0b24bc4b453c6d767970cbc', 'request_id': 'req-77b23cc5-b80a-4b44-810c-8db4451078d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.601625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0d43a9e0b24bc4b453c6d767970cbc', 'request_id': 'req-77b23cc5-b80a-4b44-810c-8db4451078d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.609322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0d43a9e0b24bc4b453c6d767970cbc', 'request_id': 'req-77b23cc5-b80a-4b44-810c-8db4451078d6'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.628616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b8eb6ddd-aed9-4e76-847c-abc1ecf0a6cf None None] MFA Rules not processed for user `56a850972bd44327b7a1328e65769b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:01.629400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] The token cache is being invalidate because project ca0d43a9e0b24bc4b453c6d767970cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:01.629894 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.630210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b8eb6ddd-aed9-4e76-847c-abc1ecf0a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.631002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.637387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1324/2648] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:01 2022] DELETE /identity/v3/projects/ca0d43a9e0b24bc4b453c6d767970cbc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:01.637387 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.638765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.679930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:01.712049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1325/2649] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:04:01.713301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.713583 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.713857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.718265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b23cc5-b80a-4b44-810c-8db4451078d6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.724181 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.725393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.725393 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.725597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] PATH_INFO: `/v3/projects/da07160c9b094e12b18b78049f608c7f` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.726785 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da07160c9b094e12b18b78049f608c7f)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.728794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.735557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:01.736969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.745338 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1326/2650] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:01.746095 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.747405 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.760830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d59a-91b3-4f6e-a970-9aa80439e906 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.765679 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.766273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.766547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.766824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.767480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.769367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7e95e-8a3f-47d3-9d42-aa8d865f0291 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.780155 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da07160c9b094e12b18b78049f608c7f', 'request_id': 'req-7e6b5753-3039-41fd-a791-75056cc625c8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.784429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:01.787085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da07160c9b094e12b18b78049f608c7f', 'request_id': 'req-7e6b5753-3039-41fd-a791-75056cc625c8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.795380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da07160c9b094e12b18b78049f608c7f', 'request_id': 'req-7e6b5753-3039-41fd-a791-75056cc625c8'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.799530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1327/2651] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:01.800790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.801070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.801354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.812547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] The token cache is being invalidate because project da07160c9b094e12b18b78049f608c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:01.812836 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.813505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.815045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1325/2652] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:01 2022] DELETE /identity/v3/projects/da07160c9b094e12b18b78049f608c7f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:01.815778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.817091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.822783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:01.823995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.882633 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b5753-3039-41fd-a791-75056cc625c8 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.885173 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1328/2653] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:01.888468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.888468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.890854 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.891594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.891983 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.892351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.893070 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.895059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.899447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009878c6-df71-4bfe-a4cc-5f866c2116c4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.905076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.905752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.906208 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.906471 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] PATH_INFO: `/v3/users/5ccb5841f6634ca6a36fca26b966b853` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.907617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccb5841f6634ca6a36fca26b966b853)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.909537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.919659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1326/2654] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:01.920630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.930504 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef724a9-a46a-4a7d-bc16-5ee72cf060f0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.935077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.935663 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.936017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.936322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.937383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:01.937714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.939557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.953761 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1327/2655] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:01.954372 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.965778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e842bb-7e86-44aa-baa6-4dd5d8a010fa admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:01.968431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccb5841f6634ca6a36fca26b966b853'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.970616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:01.971233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:01.971510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:01.971784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:01.972424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:01.974676 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:01.976727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccb5841f6634ca6a36fca26b966b853'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.977349 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccb5841f6634ca6a36fca26b966b853'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:01.985320 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1328/2656] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:01.985929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:01.987659 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:01.992941 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccb5841f6634ca6a36fca26b966b853'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.001257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a354636-842d-4c8d-b78d-e20ae6c12d4f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.002616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccb5841f6634ca6a36fca26b966b853'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.006365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.007058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.007338 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.007605 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.008212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.009322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1329/2657] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:01 2022] DELETE /identity/v3/users/5ccb5841f6634ca6a36fca26b966b853 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:02.010081 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.010242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.012082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.043802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.047417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1329/2658] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:01 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:02.051154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.092369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c2418-34d0-4eb6-afa9-a86b2f7c0ae2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.149490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c13881-23bb-4140-a55a-4aca5c0fe450 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.157643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.158530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.159140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.159497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.160815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.163777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.170763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.172699 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.173153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.173640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] PATH_INFO: `/v3/projects/b434c4e2334b4e86a00776e0b2ccd681/users/3263e2edb1134941b38123451ee12b92/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.194403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1330/2659] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:02.197977 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.202842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b434c4e2334b4e86a00776e0b2ccd681, user_id=3263e2edb1134941b38123451ee12b92, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.205672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.208342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2846530-86dc-4798-9539-23df9810319c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.213776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.214390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.214705 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.215011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.215635 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.217351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.224345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1330/2660] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:02 2022] PUT /identity/v3/projects/b434c4e2334b4e86a00776e0b2ccd681/users/3263e2edb1134941b38123451ee12b92/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:02.228376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.236187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1331/2661] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:02.241336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.250069 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=2,keystone:SELECT=90,keystone:INSERT=8 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.254585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d31ad8-43e5-4d26-84c7-0bac23a22867 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.306042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b5dfc-b6ae-4d38-a81c-eef2422f564f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.311749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.312357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.312637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.312910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.313551 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.315174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.326204 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1331/2662] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:02.329463 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.330076 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.330355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.330476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.330630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.331297 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.332882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.341641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e496c-92ce-4643-a237-d8d9555fef91 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.347047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1332/2663] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:02.347198 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.347622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.347906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.348181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] PATH_INFO: `/v3/projects/b434c4e2334b4e86a00776e0b2ccd681/users/3263e2edb1134941b38123451ee12b92/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.348717 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.352200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b434c4e2334b4e86a00776e0b2ccd681, user_id=3263e2edb1134941b38123451ee12b92, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.355004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc42c5-698b-4b99-824c-c58c2eb0edac admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.362032 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da338f-a0c6-4220-a1e9-f3d7c325d996 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.368193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1332/2664] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:02 2022] PUT /identity/v3/projects/b434c4e2334b4e86a00776e0b2ccd681/users/3263e2edb1134941b38123451ee12b92/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:02.380936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2b5f-0a68-41ef-a259-0dbc60a74d44 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.381246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2b5f-0a68-41ef-a259-0dbc60a74d44 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.381515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad2b5f-0a68-41ef-a259-0dbc60a74d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.421174 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6cad2b5f-0a68-41ef-a259-0dbc60a74d44 None None] MFA Rules not processed for user `3263e2edb1134941b38123451ee12b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:02.422934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6cad2b5f-0a68-41ef-a259-0dbc60a74d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.437210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.437899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.438172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.438439 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] PATH_INFO: `/v3/projects/41db5a582ec44bb983dc33eeb583a741/users/bb0f97faffee4018af8fea3e2c216526/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.448896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.454576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41db5a582ec44bb983dc33eeb583a741, user_id=bb0f97faffee4018af8fea3e2c216526, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.457603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.471112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1333/2665] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:02 2022] PUT /identity/v3/projects/41db5a582ec44bb983dc33eeb583a741/users/bb0f97faffee4018af8fea3e2c216526/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:02.479812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.479812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.498140 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1333/2666] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:04:02.500289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.500513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.500795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.530410 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] MFA Rules not processed for user `bb0f97faffee4018af8fea3e2c216526`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:02.532012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.548036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433011e-eca7-434f-8f0c-aa223d74fec2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.552895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.553522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.553838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.554113 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] PATH_INFO: `/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.555267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc8354c3ddd4ec79e6d20c0a717d1a4)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.557309 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.591043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.600973 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1334/2667] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:02.601993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.603274 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.610001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc8354c3ddd4ec79e6d20c0a717d1a4', 'request_id': 'req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.615399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc8354c3ddd4ec79e6d20c0a717d1a4', 'request_id': 'req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.621966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc8354c3ddd4ec79e6d20c0a717d1a4', 'request_id': 'req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.634294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] The token cache is being invalidate because project ddc8354c3ddd4ec79e6d20c0a717d1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:02.634586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.635265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:02.636857 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1334/2668] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:02 2022] DELETE /identity/v3/projects/ddc8354c3ddd4ec79e6d20c0a717d1a4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:02.637595 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.639380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.664346 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1919c58-99b3-4973-a806-f83960c7ed56 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.704788 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e17fc0-cbb7-4f62-a2cd-8885379fb271 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.710094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.710713 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.711093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.711462 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.712134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.714096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa5e272-ac39-4b95-9b6c-66d44fc29f67 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.718339 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.718950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.719248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.719511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.721254 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.740928 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.747479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1335/2669] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:02.748728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8b1e-6d57-4455-8f12-b2cb71aeaec4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.749018 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8b1e-6d57-4455-8f12-b2cb71aeaec4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.749288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed8b1e-6d57-4455-8f12-b2cb71aeaec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.769264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-6fed8b1e-6d57-4455-8f12-b2cb71aeaec4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:02.770466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-6fed8b1e-6d57-4455-8f12-b2cb71aeaec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.795686 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.797132 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.806841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1335/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:02.807717 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.820567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51147441-8e44-4a70-88fb-eda45b3bd1f6 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.842596 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1336/2671] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:02.851781 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.852067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.852339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.873992 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:02.875454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.886481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1337/2672] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:02.890524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.895796 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.896616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.896954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.897228 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.897858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.899849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.902292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e535f-5486-4a0f-bc2a-7c4992518952 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.907107 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.907710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.907991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.908263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.909294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:02.909294 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.910758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.923202 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1336/2673] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:02.924016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.924731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1338/2674] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:04:02.927285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.928487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:02.938014 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d862d-4cc6-4a51-8ade-c576c97efe69 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.942838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.943545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.943824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.944114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.944733 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.946299 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.956706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1339/2675] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:02.957304 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:02.966774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561c48c-98bf-4eae-8bba-33cbe4a7a5e1 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.971310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.971883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.972188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.972466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.973065 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:02.975925 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:02.982220 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:02.988240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca48af-5b59-4bf7-9f47-e627b443512f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:02.992542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:02.993296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:02.993683 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:02.993956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:02.996010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:03.008383 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1340/2676] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:02 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:03.009489 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.059195 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.061042 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.066491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:03.071565 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1337/2677] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:03.072726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.079970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e2b05-06ea-4a6b-bdaa-59a72b7289cf admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.084395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6aef28-f1d1-456f-b3ad-01f816348e9b service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.085722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.086936 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.087559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.088307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] PATH_INFO: `/v3/projects/61bf0372e9fd41f0a4e9c8357b162161/users/638c773db59d47bc977ab1a333fdf0c1/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.090447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.091099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.091446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.091787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.092496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.094496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.111720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] RBAC: Authorizing `identity:create_grant(project_id=61bf0372e9fd41f0a4e9c8357b162161, user_id=638c773db59d47bc977ab1a333fdf0c1, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.114559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.117820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1338/2678] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:03 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:03.122224 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.131020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1341/2679] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:03 2022] PUT /identity/v3/projects/61bf0372e9fd41f0a4e9c8357b162161/users/638c773db59d47bc977ab1a333fdf0c1/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:03.133929 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288d86c-f47f-49b4-8dc4-9d6279e51e4d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.134782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.200773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.201209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6a293-216a-468c-b516-7d3a6930c19f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.201911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.201911 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.202133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.202700 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.204536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.208146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.208846 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.209260 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.209794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.210449 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.212455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.219269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1339/2680] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:03.220666 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.233265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2f8f4-ea8f-4999-b8de-68bee80585ce admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.238604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1342/2681] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:03.239427 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.239998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.240884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.241375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.241694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.242610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.244847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.265892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1340/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:03.266200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bbbf0-b903-459a-9ff2-2a4203f7df6e admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.268011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.269907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.270809 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.271245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.271582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] PATH_INFO: `/v3/projects/61bf0372e9fd41f0a4e9c8357b162161/users/638c773db59d47bc977ab1a333fdf0c1/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.274100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61bf0372e9fd41f0a4e9c8357b162161, user_id=638c773db59d47bc977ab1a333fdf0c1, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.277384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e105a-a986-493a-8bf6-879e9ddb15f5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.281966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e23f9-a2c5-4733-ad21-05be14994545 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.288374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.289115 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.289522 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.290008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] PATH_INFO: `/v3/projects/c1a55d34adae4c8aaabdf052cb2cc2af/users/aaab300aa0e24b26b8c0c4b89cb782fb/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.307402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1343/2683] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:03 2022] PUT /identity/v3/projects/61bf0372e9fd41f0a4e9c8357b162161/users/638c773db59d47bc977ab1a333fdf0c1/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:03.310742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.311057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.311256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a55d34adae4c8aaabdf052cb2cc2af, user_id=aaab300aa0e24b26b8c0c4b89cb782fb, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.311438 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.314383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477641e3-c359-46d2-9a21-7b1209175c78 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.336078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1341/2684] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:03 2022] PUT /identity/v3/projects/c1a55d34adae4c8aaabdf052cb2cc2af/users/aaab300aa0e24b26b8c0c4b89cb782fb/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:03.349082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] MFA Rules not processed for user `638c773db59d47bc977ab1a333fdf0c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:03.350300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:03.350903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.351418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.351707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.388011 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] MFA Rules not processed for user `aaab300aa0e24b26b8c0c4b89cb782fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:03.389846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:03.406956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:03.444051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1344/2685] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:03.456351 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.485676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1342/2686] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:04:03.503868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:03.503868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:03.527922 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d235a-91ff-4c3a-af24-4611cdfb99c0 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.532424 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.533035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.533316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.533585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.535408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:03.551330 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:03.575954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441568f1-bd5e-4a16-ace5-4ccfd9db5df2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:03.579527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:03.580128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:03.580399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:03.580668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:03.589721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.591873 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.598239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:03.599696 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b25e42-8639-4a65-a7bf-3b40d869d310 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:03.602010 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1343/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:03.608995 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1345/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:04:06.197441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:06.268194 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8f1b1-f4b8-43e9-90e2-a939fa69ade8 service glance] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:06.273235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:06.273863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:06.274143 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:06.274425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] PATH_INFO: `/v3/users/bb0f97faffee4018af8fea3e2c216526` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:06.281100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0f97faffee4018af8fea3e2c216526)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:06.281100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:06.339308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0f97faffee4018af8fea3e2c216526'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.347809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0f97faffee4018af8fea3e2c216526'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.348521 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0f97faffee4018af8fea3e2c216526'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.361056 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0f97faffee4018af8fea3e2c216526'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.372101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0f97faffee4018af8fea3e2c216526'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.379079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1344/2689] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:06 2022] DELETE /identity/v3/users/bb0f97faffee4018af8fea3e2c216526 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:06.383813 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:06.384355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:06.384488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:06.407043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:06.408230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:06.490204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1346/2690] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:06.496342 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:06.497727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:06.566678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf88c6-02e2-4ff0-8d6a-14ded8dcb790 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:06.570371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:06.571002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:06.571283 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:06.571554 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:06.581669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:06.583121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:06.587119 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:06.595188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1345/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:06.888102 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:06.889689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:06.902654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df783fba-e847-4e8f-9b4d-9fe6302d71c9 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:06.907475 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:06.908313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:06.908638 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:06.908923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] PATH_INFO: `/v3/projects/41db5a582ec44bb983dc33eeb583a741` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:06.910121 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41db5a582ec44bb983dc33eeb583a741)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:06.912450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:06.944349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:06.960953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41db5a582ec44bb983dc33eeb583a741', 'request_id': 'req-356d906c-985e-453d-8437-03e1b6d65224'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.969965 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41db5a582ec44bb983dc33eeb583a741', 'request_id': 'req-356d906c-985e-453d-8437-03e1b6d65224'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.977055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41db5a582ec44bb983dc33eeb583a741', 'request_id': 'req-356d906c-985e-453d-8437-03e1b6d65224'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.992735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] The token cache is being invalidate because project 41db5a582ec44bb983dc33eeb583a741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:06.993022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.993662 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:06.995266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1347/2692] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:06 2022] DELETE /identity/v3/projects/41db5a582ec44bb983dc33eeb583a741 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:07.000016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:07.010771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad6832-887b-495b-b838-ee730c0cf622 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:07.075530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:07.076169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.076495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.076707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] PATH_INFO: `/v3/users/638c773db59d47bc977ab1a333fdf0c1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.077819 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=638c773db59d47bc977ab1a333fdf0c1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:07.081026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:07.130536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638c773db59d47bc977ab1a333fdf0c1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.137872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638c773db59d47bc977ab1a333fdf0c1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.138760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638c773db59d47bc977ab1a333fdf0c1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.154448 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638c773db59d47bc977ab1a333fdf0c1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.161713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7a13a54e-f3f4-4789-bde4-e5561fd885b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638c773db59d47bc977ab1a333fdf0c1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.168397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1346/2693] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:06 2022] DELETE /identity/v3/users/638c773db59d47bc977ab1a333fdf0c1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:07.457311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:07.458793 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:07.472904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d906c-985e-453d-8437-03e1b6d65224 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:07.552365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:07.553801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.553908 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.554090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] PATH_INFO: `/v3/projects/61bf0372e9fd41f0a4e9c8357b162161` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.555292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61bf0372e9fd41f0a4e9c8357b162161)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:07.557198 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:07.617095 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61bf0372e9fd41f0a4e9c8357b162161', 'request_id': 'req-a8546484-df86-4122-b36e-5ba23a47df6a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.623916 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61bf0372e9fd41f0a4e9c8357b162161', 'request_id': 'req-a8546484-df86-4122-b36e-5ba23a47df6a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.630880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61bf0372e9fd41f0a4e9c8357b162161', 'request_id': 'req-a8546484-df86-4122-b36e-5ba23a47df6a'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.646288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] The token cache is being invalidate because project 61bf0372e9fd41f0a4e9c8357b162161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:07.646798 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.647509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-a8546484-df86-4122-b36e-5ba23a47df6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:07.651006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1348/2694] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:07 2022] DELETE /identity/v3/projects/61bf0372e9fd41f0a4e9c8357b162161 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:07.654789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.655906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.656336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.678997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:07.681534 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:07.776388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1347/2695] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:07.778454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.778694 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.779041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.802758 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:07.804084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:07.817875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1349/2696] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:07.818909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:07.829849 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53571534-7489-4599-8eba-d2c8729ea1f0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:07.834964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:07.835723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.835993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.836272 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.836977 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:07.837238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:07.838884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:07.856488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:07.859277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1348/2697] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:07.867265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661005b-5d54-466a-a3ad-1239bb6e4498 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:07.887926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:07.888531 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:07.888913 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:07.889437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:07.890211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:07.892862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:07.947259 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1350/2698] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:07 2022] POST /identity/v3/projects => generated 371 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:07.952499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:07.965515 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:07.971557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48234844-50b1-478b-9d63-3005f79c8f1d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.078992 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.084799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.085088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.085841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.086486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.088655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.135120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1349/2699] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:07 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:08.138455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.152879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb990c81-4cb2-42e0-9025-7ddc409c29ec admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.158069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.158369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.159006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.159346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.159621 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.160255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.160379 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.161886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.173747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386622d2-7190-46a3-87a2-8b9d76142425 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.173933 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1351/2700] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:08.177687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.179022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.179438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.179721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.179998 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] PATH_INFO: `/v3/users/56a850972bd44327b7a1328e65769b9e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.181131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a850972bd44327b7a1328e65769b9e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.183082 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.190239 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451a3e6-9105-4e51-9991-d2a53011d841 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.195645 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.196249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.196524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.196794 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.197606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.199283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.201094 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:08.219154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1352/2701] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:08.221100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.232863 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1c0ea-381d-4dfa-9e67-07f0bd9fb4c2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.243458 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.244552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.244833 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.245254 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] PATH_INFO: `/v3/projects/0aa39f9ffa094aea95f5c5a9299880aa/users/994d8a9fcee246aaa4b7357bdb3abf52/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.264893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a850972bd44327b7a1328e65769b9e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.266175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa39f9ffa094aea95f5c5a9299880aa, user_id=994d8a9fcee246aaa4b7357bdb3abf52, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.269483 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e17b5-394a-4c6a-bc79-2f09214e7fd2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.272680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a850972bd44327b7a1328e65769b9e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.273323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a850972bd44327b7a1328e65769b9e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.288618 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1353/2702] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:08 2022] PUT /identity/v3/projects/0aa39f9ffa094aea95f5c5a9299880aa/users/994d8a9fcee246aaa4b7357bdb3abf52/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:08.295356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a850972bd44327b7a1328e65769b9e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.300256 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.300585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.300923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.306160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4dbd27aa-47f6-42be-b6b2-28cf13d00dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a850972bd44327b7a1328e65769b9e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.314340 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1350/2703] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:08 2022] DELETE /identity/v3/users/56a850972bd44327b7a1328e65769b9e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:08.320336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.320844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.321121 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.341041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] MFA Rules not processed for user `994d8a9fcee246aaa4b7357bdb3abf52`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:08.342273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.342495 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:08.343711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.418541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1351/2704] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:08.427009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1354/2705] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:04:08.430180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.432537 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.436160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.437577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.446131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:08.498523 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b27d44-3e50-42ef-b6d7-477557a87b73 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.499351 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970e48a-14ac-4c7c-aba8-6b8db6d3d933 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.505829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.506469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.506748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.507053 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.514197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.515124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.515411 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.515907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.518052 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.519756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.525026 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.526488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.530256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1352/2706] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:08.535413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1355/2707] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:04:08.822944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.824943 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:08.844386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb4f8-24ec-4a45-9cd5-d544f513ddf5 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:08.848993 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:08.849945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:08.851270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:08.851752 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] PATH_INFO: `/v3/projects/368b233309724d538d4756ec424f15df` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:08.853869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] RBAC: Authorizing `identity:delete_project(project_id=368b233309724d538d4756ec424f15df)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:08.855942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:08.909384 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368b233309724d538d4756ec424f15df', 'request_id': 'req-dc52fa00-464a-436c-b542-65ce626b3add'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.917899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368b233309724d538d4756ec424f15df', 'request_id': 'req-dc52fa00-464a-436c-b542-65ce626b3add'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.925584 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368b233309724d538d4756ec424f15df', 'request_id': 'req-dc52fa00-464a-436c-b542-65ce626b3add'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.936604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] The token cache is being invalidate because project 368b233309724d538d4756ec424f15df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:08.936894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.937621 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:08.939419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1353/2708] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:08 2022] DELETE /identity/v3/projects/368b233309724d538d4756ec424f15df => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:08.945042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:08.958281 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e72a9-7b4c-4f30-a0c9-3b0c22a48652 service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.070166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.072290 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.072984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.072984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] PATH_INFO: `/v3/users/3263e2edb1134941b38123451ee12b92` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.074161 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3263e2edb1134941b38123451ee12b92)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.076950 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.082361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:09.130244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3263e2edb1134941b38123451ee12b92'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.137827 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3263e2edb1134941b38123451ee12b92'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.138470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3263e2edb1134941b38123451ee12b92'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.145233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3263e2edb1134941b38123451ee12b92'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.157619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-5ba7fc00-2475-4c8b-8b92-faaef5de257b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3263e2edb1134941b38123451ee12b92'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.163369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1356/2709] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:08 2022] DELETE /identity/v3/users/3263e2edb1134941b38123451ee12b92 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:09.465329 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:09.467125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:09.482505 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52fa00-464a-436c-b542-65ce626b3add admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.547933 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.548566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.548850 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.549122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] PATH_INFO: `/v3/projects/b434c4e2334b4e86a00776e0b2ccd681` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.550246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b434c4e2334b4e86a00776e0b2ccd681)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.552210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.589366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:09.603435 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b434c4e2334b4e86a00776e0b2ccd681', 'request_id': 'req-10f01ede-0aa1-4270-a431-322799fc8240'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.611566 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b434c4e2334b4e86a00776e0b2ccd681', 'request_id': 'req-10f01ede-0aa1-4270-a431-322799fc8240'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.618610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b434c4e2334b4e86a00776e0b2ccd681', 'request_id': 'req-10f01ede-0aa1-4270-a431-322799fc8240'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.634057 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] The token cache is being invalidate because project b434c4e2334b4e86a00776e0b2ccd681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:09.634375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.635211 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-10f01ede-0aa1-4270-a431-322799fc8240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:09.637229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1354/2710] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:09 2022] DELETE /identity/v3/projects/b434c4e2334b4e86a00776e0b2ccd681 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:09.642062 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.642543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.642644 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.662288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:09.663507 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:09.723058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1357/2711] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:09.730570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.730852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.731152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.750323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:09.751815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:09.759671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1355/2712] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:09.764425 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:09.774837 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de8f3-e02e-4ba8-af8f-db307db7676f None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.780381 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.781104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.781359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.781808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.782650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:09.783109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.785311 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.797753 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1358/2713] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:04:09.802267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:09.813602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083b162-94a5-40c1-b994-bbdc171376ef None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.818452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.819088 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.819381 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.819649 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.820264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.822157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.849350 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1356/2714] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:09.853529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:09.863568 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d323943-507a-47d0-a393-b34384a6b85a admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.923679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.924302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.924576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.924841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.925465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.927408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.949186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:09.955820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1359/2715] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:09 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:09.959845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:09.970694 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184f696-2444-4b25-b6d1-06307d40cda0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:09.975289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:09.975823 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:09.976097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:09.976355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:09.977016 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:09.978593 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:09.989429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1357/2716] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:09.993975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:10.006193 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca04c7-e7fa-45cc-8151-99fb67d2aae2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:10.011341 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:10.012059 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:10.012441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:10.012783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:10.013447 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:10.015067 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13ac28-9eae-46a3-ab15-0a6f005c7058 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:10.026190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1360/2717] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:10.031054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:10.041263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1456-795c-4e35-b553-ebda6e8ae682 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:10.046727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:10.047398 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:10.047677 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:10.048065 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] PATH_INFO: `/v3/projects/6a4be6a1587d4eaf857b66bfd37fb835/users/ba7ec38b775d485788512cfe0174a9fe/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:10.079301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a4be6a1587d4eaf857b66bfd37fb835, user_id=ba7ec38b775d485788512cfe0174a9fe, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:10.083486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:10.099426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1358/2718] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:10 2022] PUT /identity/v3/projects/6a4be6a1587d4eaf857b66bfd37fb835/users/ba7ec38b775d485788512cfe0174a9fe/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:10.112706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743966-64f9-4e64-93c2-934474cf26df None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:10.112706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743966-64f9-4e64-93c2-934474cf26df None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:10.113188 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51743966-64f9-4e64-93c2-934474cf26df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:10.145982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-51743966-64f9-4e64-93c2-934474cf26df None None] MFA Rules not processed for user `ba7ec38b775d485788512cfe0174a9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:10.147230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-51743966-64f9-4e64-93c2-934474cf26df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:10.212637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1361/2719] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:10.224002 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:10.225766 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:10.286409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff0ace-da44-4869-be71-1d6ce0317b89 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:10.290586 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:10.291324 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:10.291607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:10.291935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:10.301376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:10.302841 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:10.313282 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1359/2720] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:16.941410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51743966-64f9-4e64-93c2-934474cf26df None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:16.943112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-51743966-64f9-4e64-93c2-934474cf26df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:17.041719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51743966-64f9-4e64-93c2-934474cf26df None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:17.047538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:17.048141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:17.048419 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:17.048691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:17.050450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:17.063660 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:17.065253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ecde-263a-40bb-9ca8-396f9f2fefbf service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:17.078090 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1362/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:20.307769 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:21.117069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:21.198139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ada4a7-e8c6-4672-8272-6bd344d96040 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:21.203386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:21.204044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:21.204321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:21.204590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] PATH_INFO: `/v3/users/ba7ec38b775d485788512cfe0174a9fe` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:21.205743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7ec38b775d485788512cfe0174a9fe)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:21.207722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:21.256295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7ec38b775d485788512cfe0174a9fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.265921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7ec38b775d485788512cfe0174a9fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.266779 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7ec38b775d485788512cfe0174a9fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.281957 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7ec38b775d485788512cfe0174a9fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.289157 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7ec38b775d485788512cfe0174a9fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.295921 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1360/2722] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:21 2022] DELETE /identity/v3/users/ba7ec38b775d485788512cfe0174a9fe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:21.300407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:21.300862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:21.300970 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:21.319911 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:21.321140 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:21.366323 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:21.383802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1363/2723] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:21.393620 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:21.395068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:21.477178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b1a1-2f6d-4426-84d3-16024e60729a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:21.482740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:21.483426 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:21.483733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:21.484004 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:21.493790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:21.495257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e75fa-862d-4baa-88a4-eb9ab633eaff service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:21.505473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1361/2724] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:21.809015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:21.810446 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:21.823128 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9b53f-1d45-4fd7-91e3-7f483aa093f4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:21.828558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:21.829326 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:21.829672 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:21.830011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] PATH_INFO: `/v3/projects/6a4be6a1587d4eaf857b66bfd37fb835` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:21.831266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a4be6a1587d4eaf857b66bfd37fb835)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:21.833182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:21.883087 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a4be6a1587d4eaf857b66bfd37fb835', 'request_id': 'req-e9b1884c-35f4-428a-8bc5-dac13bd60562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.889233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a4be6a1587d4eaf857b66bfd37fb835', 'request_id': 'req-e9b1884c-35f4-428a-8bc5-dac13bd60562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.895543 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a4be6a1587d4eaf857b66bfd37fb835', 'request_id': 'req-e9b1884c-35f4-428a-8bc5-dac13bd60562'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.911108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] The token cache is being invalidate because project 6a4be6a1587d4eaf857b66bfd37fb835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:21.911395 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.912056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e9b1884c-35f4-428a-8bc5-dac13bd60562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:21.913547 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1364/2725] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:21 2022] DELETE /identity/v3/projects/6a4be6a1587d4eaf857b66bfd37fb835 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:21.920507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:21.920507 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:21.920947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:21.931607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:21.948570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:21.949797 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:22.014945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1362/2726] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:22.022871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.023182 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.023469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.043839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:22.044941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:22.052516 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1365/2727] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:22.056926 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.068610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7db0aa-44e3-40c1-828e-51589c6e927f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.074453 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.074712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.075020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.075296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.075962 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:22.076258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.078025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.094215 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1363/2728] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:22.098521 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.118826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b42029-7968-4803-ad45-75319d466b9e None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.118826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.118826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.118826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.119300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.119300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.119300 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.144831 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1366/2729] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:22 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:04:22.150212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.161348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33714d-b680-4cdd-95e8-e9e6b095b349 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.232800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.233425 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.233721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.233979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.234618 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.236617 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.260438 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1364/2730] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:22 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:04:22.264538 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.275027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb43da7-8b76-4875-977a-2632f716b5c4 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.280079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.280762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.281040 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.281308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.281941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.283617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.296129 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1367/2731] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:22.300419 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.310789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce4bda-caa0-45a7-8818-74d9ee416ebf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.315572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.316164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.316441 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.316709 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.317345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.318930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fb6db-f8eb-46d2-8507-91fcf6234af8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.330355 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1365/2732] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:22.334754 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.345190 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950e74-4adf-4617-ab04-7182e1158964 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.349742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.350378 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.350654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.350920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] PATH_INFO: `/v3/projects/2921854d37164db489c55a402b111407/users/7c1dd45e4de74ec7803d20c2aadfa61d/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.365101 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2921854d37164db489c55a402b111407, user_id=7c1dd45e4de74ec7803d20c2aadfa61d, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.367799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.381365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1368/2733] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:22 2022] PUT /identity/v3/projects/2921854d37164db489c55a402b111407/users/7c1dd45e4de74ec7803d20c2aadfa61d/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:22.399257 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.399776 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.399917 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.430542 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] MFA Rules not processed for user `7c1dd45e4de74ec7803d20c2aadfa61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:22.431816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:22.490265 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:22.498780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1366/2734] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:22.511711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:22.513402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:22.576626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdd4fc-34e9-476b-a501-48e3f63b4122 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:22.580150 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:22.580744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:22.581023 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:22.581287 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:22.590092 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:22.591554 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e05a8-e937-4593-9315-763963698ca3 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:22.600027 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1369/2735] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:04:32.498955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:32.596433 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:40.419923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:40.422626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:40.515589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fa163-bddb-41a6-86bb-3de975482142 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:40.526513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:40.527439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:40.527891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:40.528093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] PATH_INFO: `/v3/users/aaab300aa0e24b26b8c0c4b89cb782fb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:40.529604 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaab300aa0e24b26b8c0c4b89cb782fb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:40.531893 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:40.581226 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaab300aa0e24b26b8c0c4b89cb782fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:40.589148 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaab300aa0e24b26b8c0c4b89cb782fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:40.589787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaab300aa0e24b26b8c0c4b89cb782fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:40.602599 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaab300aa0e24b26b8c0c4b89cb782fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:40.609363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaab300aa0e24b26b8c0c4b89cb782fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:40.615938 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1367/2736] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:40 2022] DELETE /identity/v3/users/aaab300aa0e24b26b8c0c4b89cb782fb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:40.620333 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:40.620851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:40.620851 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:40.639642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:40.641064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:40.698058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1370/2737] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:40.707936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:40.709341 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:40.768133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce7c0-2d5e-43de-8022-b823d8191faf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:40.772023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:40.772627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:40.772906 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:40.773179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:40.783248 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:40.784680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6652cd-618c-4aed-a611-fe44c76f23a6 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:40.794167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1368/2738] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:41.059056 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.059886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:41.072342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5f87d-01e0-4396-98e8-c071e33bb9b1 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.076944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.077730 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.078123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.078391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] PATH_INFO: `/v3/projects/c1a55d34adae4c8aaabdf052cb2cc2af` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.080017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a55d34adae4c8aaabdf052cb2cc2af)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.081910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.129943 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a55d34adae4c8aaabdf052cb2cc2af', 'request_id': 'req-65897e03-b29c-4613-9f16-c683a172d13f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:41.135391 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a55d34adae4c8aaabdf052cb2cc2af', 'request_id': 'req-65897e03-b29c-4613-9f16-c683a172d13f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:41.141506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a55d34adae4c8aaabdf052cb2cc2af', 'request_id': 'req-65897e03-b29c-4613-9f16-c683a172d13f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:41.154123 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] The token cache is being invalidate because project c1a55d34adae4c8aaabdf052cb2cc2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:41.154407 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:41.155047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65897e03-b29c-4613-9f16-c683a172d13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:41.156488 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1371/2739] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:41 2022] DELETE /identity/v3/projects/c1a55d34adae4c8aaabdf052cb2cc2af => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:41.162013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.162295 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.162562 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.181446 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:41.182639 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:41.199865 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:41.243246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1369/2740] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:41.252742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.252960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.253230 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.273344 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:41.274544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:41.282312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1372/2741] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:41.286632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.296422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d825c-b724-457d-b01d-73fdfbb46c69 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.301396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.302031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.302305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.302572 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.303255 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:41.303559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.305756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.320139 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1370/2742] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:41.324582 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.334392 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec417f5d-0198-40fb-93ef-0b6d4f0933b4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.338951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.339562 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.339842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.340117 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.340731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.342619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.369249 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1373/2743] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:41 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:41.373360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.384468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ed52d-8c4a-4d99-bb20-e5485de9d0be admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.441309 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.442030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.442307 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.442577 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.443323 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.445499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.466831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1371/2744] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:41 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:41.471490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.481898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43604f10-f04f-40cb-80ed-158980d98e8f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.486348 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.486926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.487236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.487502 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.488130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.489702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.503435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1374/2745] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:04:41.505449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.515548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebefd0-b0f3-4eea-af17-66a438b44d7f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.524252 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.524847 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.525122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.525389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.526026 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.527637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac414ef-f578-4f8b-8434-34dbcb614b13 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.537858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1372/2746] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:41.542269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.553558 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e90f6-ae37-4b02-af33-7dad693a3a82 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.558435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.559124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.559406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.559679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] PATH_INFO: `/v3/projects/50c3e414e6dc4eceb7c7e37426f25cb1/users/5962a6cc3af54010ab9b5963970c0f57/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.574835 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c3e414e6dc4eceb7c7e37426f25cb1, user_id=5962a6cc3af54010ab9b5963970c0f57, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.577486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.590081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1375/2747] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:41 2022] PUT /identity/v3/projects/50c3e414e6dc4eceb7c7e37426f25cb1/users/5962a6cc3af54010ab9b5963970c0f57/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:41.600633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.600913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.601188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.629359 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] MFA Rules not processed for user `5962a6cc3af54010ab9b5963970c0f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:41.630553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:41.693802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1373/2748] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:41.704370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:41.706211 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:41.728791 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:41.784961 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900602f-332e-4eed-a8fa-36172bb03936 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:41.788760 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:41.789360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:41.789639 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:41.789910 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:41.799020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:41.800459 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49de31-db9c-4b56-9ade-97f9563b9165 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:41.809980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1376/2749] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:51.691458 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:51.724468 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:51.725888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:51.791722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57fcbe-4cc5-4331-bd7c-73929ee05d6f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:51.796370 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:51.796989 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:51.797269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:51.797543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] PATH_INFO: `/v3/users/7c1dd45e4de74ec7803d20c2aadfa61d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:51.798702 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1dd45e4de74ec7803d20c2aadfa61d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:51.800698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:51.806314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=26 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:51.857763 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1dd45e4de74ec7803d20c2aadfa61d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:51.872969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1dd45e4de74ec7803d20c2aadfa61d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:51.873629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1dd45e4de74ec7803d20c2aadfa61d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:51.886838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1dd45e4de74ec7803d20c2aadfa61d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:51.894678 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1dd45e4de74ec7803d20c2aadfa61d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:51.901538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1374/2750] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:51 2022] DELETE /identity/v3/users/7c1dd45e4de74ec7803d20c2aadfa61d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:51.906066 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:51.906352 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:51.906636 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:51.926767 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:51.927841 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:52.017544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1377/2751] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:52.028784 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.031294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:52.106591 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70b422-d2ce-48f3-8ade-9c050e33cf37 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.111269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.112070 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.112356 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.112627 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.124012 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.125478 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8d935-9e26-4aa1-ad8c-8bcadda883d1 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.140959 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1375/2752] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:52.406291 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.407939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:52.427609 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d998b-8d3e-458f-ab10-db9cb201c92b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.432431 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.433252 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.433470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.433736 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] PATH_INFO: `/v3/projects/2921854d37164db489c55a402b111407` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.435359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2921854d37164db489c55a402b111407)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.437272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.483454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2921854d37164db489c55a402b111407', 'request_id': 'req-042d76dd-61b6-4725-981e-a7d8dd4f784e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:52.489679 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2921854d37164db489c55a402b111407', 'request_id': 'req-042d76dd-61b6-4725-981e-a7d8dd4f784e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:52.495867 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2921854d37164db489c55a402b111407', 'request_id': 'req-042d76dd-61b6-4725-981e-a7d8dd4f784e'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:52.508302 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] The token cache is being invalidate because project 2921854d37164db489c55a402b111407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:52.508587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:52.509207 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-042d76dd-61b6-4725-981e-a7d8dd4f784e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:52.510683 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1378/2753] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:52 2022] DELETE /identity/v3/projects/2921854d37164db489c55a402b111407 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:52.515496 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.515787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.516063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.535459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:52.536653 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:52.553640 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:52.598669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1376/2754] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:52.608587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.608989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.609410 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.631082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:52.632266 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:52.640043 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1379/2755] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:52.644603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.654695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286969b-2799-4af5-92b2-9d0947f1e1ea None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.660069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.660744 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.661035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.661399 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.661958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:52.662266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.663894 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.683440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1377/2756] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:52.684625 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.694061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62797e9-09a5-4eb3-baff-47ef3141c1d4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.698832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.699465 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.699745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.700016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.700630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.702564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.732711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1380/2757] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:52 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:52.736898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.747285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee153a-d586-4b6d-bbdd-1f2b18eaa915 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.804447 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.805050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.805327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.805602 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.806238 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.808193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.829217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1378/2758] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:52 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:04:52.834049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.844146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44ce47-88c1-4c86-bc91-9ff12dbc38f7 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.849413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.850008 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.850285 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.850552 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.851245 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.852853 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.862657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1381/2759] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:52.867191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.877368 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc082455-f7eb-4da8-a31b-ea3968eb3134 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.882881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.883543 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.883802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.884073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.884706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.886297 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cb38a-1cdd-427c-abfa-2d711fbb2f86 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.896815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1379/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:52.901055 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:52.911154 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ca767-135b-4d3b-9a97-705c3057c501 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:52.915642 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:52.916274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.916549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.916817 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] PATH_INFO: `/v3/projects/eb8ebe32cb054a8dbc6fab289094bf70/users/b0dfec2171f9467598ce003a119e1616/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.932479 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8ebe32cb054a8dbc6fab289094bf70, user_id=b0dfec2171f9467598ce003a119e1616, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:52.935153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:52.947745 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1382/2761] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:52 2022] PUT /identity/v3/projects/eb8ebe32cb054a8dbc6fab289094bf70/users/b0dfec2171f9467598ce003a119e1616/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:52.957972 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:52.958262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:52.958539 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:52.985460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] MFA Rules not processed for user `b0dfec2171f9467598ce003a119e1616`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:52.986674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:53.051256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1380/2762] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:53.070137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:53.071865 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:53.093328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:53.131255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0862c2-78f1-409e-bded-85c2bf054cdd admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:53.134899 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:53.135529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:53.135812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:53.136083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:53.144474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:53.145886 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5eeb87-aab5-43a4-b961-6580ae87c6f2 service cinder] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:53.155380 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1383/2763] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:55.859875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:55.861616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:55.885217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:55.932934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed0b4d-9212-4cf4-b83e-61af7fe4e29a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:55.937386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:55.938005 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:55.938279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:55.938549 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] PATH_INFO: `/v3/users/5962a6cc3af54010ab9b5963970c0f57` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:55.939688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5962a6cc3af54010ab9b5963970c0f57)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:55.941601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:55.989048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5962a6cc3af54010ab9b5963970c0f57'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:55.996072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5962a6cc3af54010ab9b5963970c0f57'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:55.996692 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5962a6cc3af54010ab9b5963970c0f57'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.008868 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5962a6cc3af54010ab9b5963970c0f57'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.015597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5962a6cc3af54010ab9b5963970c0f57'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.022176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1381/2764] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:55 2022] DELETE /identity/v3/users/5962a6cc3af54010ab9b5963970c0f57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:56.026698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.027250 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.027359 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.046509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:56.047742 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:56.108776 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1384/2765] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:56.118523 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.119965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:56.183697 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae51b2ad-7070-4743-b1b5-e7dd149f8469 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.187328 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:56.187930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.188205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.188469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.198360 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:56.199802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6256b-fd38-43ec-9eab-10999d90819f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:56.209848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1382/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:04:56.509966 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.511838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:56.524403 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14684b-027e-45e2-b740-8ffcce815458 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.529497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:56.530168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.530448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.530735 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] PATH_INFO: `/v3/projects/50c3e414e6dc4eceb7c7e37426f25cb1` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.532033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c3e414e6dc4eceb7c7e37426f25cb1)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:56.533987 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:56.597826 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c3e414e6dc4eceb7c7e37426f25cb1', 'request_id': 'req-70403d8f-92d5-4792-90d5-4f76ce776880'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.604041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c3e414e6dc4eceb7c7e37426f25cb1', 'request_id': 'req-70403d8f-92d5-4792-90d5-4f76ce776880'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.611133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c3e414e6dc4eceb7c7e37426f25cb1', 'request_id': 'req-70403d8f-92d5-4792-90d5-4f76ce776880'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.628408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] The token cache is being invalidate because project 50c3e414e6dc4eceb7c7e37426f25cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:56.628731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.630109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-70403d8f-92d5-4792-90d5-4f76ce776880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:56.632199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1385/2767] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:56 2022] DELETE /identity/v3/projects/50c3e414e6dc4eceb7c7e37426f25cb1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:56.641127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.641408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.641739 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.661836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:56.663093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:56.700250 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:56.728423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1383/2768] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:56.737136 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.737534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.737810 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.766321 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:56.767544 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:56.776237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1386/2769] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:56.780605 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.790795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98a323-e1a0-485c-b6ea-b989cb800c94 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.795550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:56.799229 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.799601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.800025 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.801304 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:56.801663 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:56.803408 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:56.815529 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1384/2770] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:56.821799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.831591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de941db-f40d-41a6-bdbb-df88c8771a52 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.836421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:56.837009 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.837283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.837551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.838164 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:56.840114 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:56.874599 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1387/2771] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:56.875780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.886623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4d216-265b-45be-9f86-69844469ff59 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.953616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:56.954348 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:56.954629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:56.954903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:56.955585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:56.957540 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:56.979939 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1385/2772] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:56 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:04:56.983892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:56.994875 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cce1d-4eb2-4ea3-9c9d-9c91458d285d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:56.999848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:57.000539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:57.000926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:57.001298 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:57.002265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:57.003980 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:57.014011 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1388/2773] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:57.018076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:57.029023 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c4a3-5c02-4f1a-a512-d3c1b4f27657 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:57.034665 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:57.035844 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:57.036246 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:57.036675 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:57.037518 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:57.040137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeefa14-4e77-43e9-98f6-244177376aa1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:57.054527 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1386/2774] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:57.058940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:57.072739 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153a68-b331-4cfb-adf2-fa18d71a0d06 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:57.077948 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:57.078670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:57.078954 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:57.079255 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] PATH_INFO: `/v3/projects/d6b4958df154416a820335d272cec49d/users/8f8ef87496bb43d1bab7c5f6f101d6aa/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:57.081677 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:57.099328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b4958df154416a820335d272cec49d, user_id=8f8ef87496bb43d1bab7c5f6f101d6aa, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:57.102033 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:57.115989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1389/2775] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:57 2022] PUT /identity/v3/projects/d6b4958df154416a820335d272cec49d/users/8f8ef87496bb43d1bab7c5f6f101d6aa/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:04:57.120987 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:57.121273 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:57.121547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:57.143816 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:57.145068 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:57.207954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1387/2776] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:04:57.217878 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:57.219695 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:57.287421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d988d89-4f0e-4481-a829-c4526f9619fe admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:57.291135 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:57.291734 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:57.292013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:57.292282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:57.301349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:57.302805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426fc2f-a161-4c83-9bb4-c4c634557d4e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:57.312670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1390/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:58.315429 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:58.317423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:58.329322 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2ddc9-ad18-4db7-bbb3-6bc2536a1ff3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:58.334780 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:58.335553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:58.335912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:58.336241 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] PATH_INFO: `/v3/users/b0dfec2171f9467598ce003a119e1616` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:58.337669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0dfec2171f9467598ce003a119e1616)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:58.340159 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:58.393371 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:58.403452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0dfec2171f9467598ce003a119e1616'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:58.416695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0dfec2171f9467598ce003a119e1616'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:58.417876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0dfec2171f9467598ce003a119e1616'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:58.434191 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0dfec2171f9467598ce003a119e1616'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:58.442582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0dfec2171f9467598ce003a119e1616'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:58.450485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1388/2778] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:04:58 2022] DELETE /identity/v3/users/b0dfec2171f9467598ce003a119e1616 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:58.455201 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:58.455716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:58.455962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:58.477972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:58.479279 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:58.548789 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1391/2779] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:58.560651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:58.562530 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:58.660676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88364498-447c-473b-aabc-0e441e203efd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:58.664835 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:58.665606 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:58.666114 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:58.666251 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:58.695733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:58.697659 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43078feb-1f8c-4cb1-bf8b-9d095838d347 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:58.713357 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1389/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:04:59.074166 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.076514 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:59.089678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aceb4-126a-4078-b2e0-3e2271bb6313 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.094615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.095482 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.095982 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.096469 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] PATH_INFO: `/v3/projects/eb8ebe32cb054a8dbc6fab289094bf70` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.097972 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8ebe32cb054a8dbc6fab289094bf70)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.100313 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.157339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8ebe32cb054a8dbc6fab289094bf70', 'request_id': 'req-d8a6bab5-979c-42e4-89df-b6df1c048b96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:59.163576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8ebe32cb054a8dbc6fab289094bf70', 'request_id': 'req-d8a6bab5-979c-42e4-89df-b6df1c048b96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:59.169905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8ebe32cb054a8dbc6fab289094bf70', 'request_id': 'req-d8a6bab5-979c-42e4-89df-b6df1c048b96'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:59.185650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] The token cache is being invalidate because project eb8ebe32cb054a8dbc6fab289094bf70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:04:59.185940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:59.186561 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d8a6bab5-979c-42e4-89df-b6df1c048b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:04:59.188077 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1392/2781] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:04:59 2022] DELETE /identity/v3/projects/eb8ebe32cb054a8dbc6fab289094bf70 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:59.193768 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.194047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.194333 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.214243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:59.215397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:59.282020 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1390/2782] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:59.291616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.291904 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.292242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.314195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:59.315779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:59.327522 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1393/2783] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:59.333482 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.345798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd713f2-c0f1-4232-869b-67c97f609c47 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.353074 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.354117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.354726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.355253 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.356327 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:04:59.356809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.360336 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.374934 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1391/2784] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:04:59.380421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.385214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:59.392047 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393386a-2546-4ac6-8002-5eafe7f4b9a2 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.399699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.400400 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.401532 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.402108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.403035 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.405548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.449777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1394/2785] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:04:59 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:59.451134 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.464063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64571768-eb8b-49e5-a775-cfd82c6d61c2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.524818 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:04:59.542318 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.543178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.543462 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.543733 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.544367 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.546706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.577789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1392/2786] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:04:59 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:04:59.582773 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.594594 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d056-9e3f-4cf6-9132-4fb19227bc3b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.600037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.600650 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.600928 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.601200 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.601848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.603900 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.615790 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1395/2787] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:59.620279 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.631974 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76216fe8-5d13-46b3-bf50-155e272f7f04 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.637980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.638596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.638861 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.639176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.639838 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.641609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d5e90-c576-4b85-943b-b4a7207458f1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.677144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1393/2788] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:04:59.681680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.700802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397ad9-1d77-4fbd-b906-b9e3b4834b6d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.706569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.707848 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.708312 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.708584 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] PATH_INFO: `/v3/projects/3fdb6c1641c74f97a10f57644b87197b/users/76b5d1c7c9f24c2e981532b5d6097d66/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.725722 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fdb6c1641c74f97a10f57644b87197b, user_id=76b5d1c7c9f24c2e981532b5d6097d66, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:04:59.728466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:04:59.744998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1396/2789] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:04:59 2022] PUT /identity/v3/projects/3fdb6c1641c74f97a10f57644b87197b/users/76b5d1c7c9f24c2e981532b5d6097d66/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:04:59.756608 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6d4-a7cb-439c-9537-427cb3d340cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.757030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6d4-a7cb-439c-9537-427cb3d340cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.757719 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6d4-a7cb-439c-9537-427cb3d340cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:04:59.803129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7415a6d4-a7cb-439c-9537-427cb3d340cf None None] MFA Rules not processed for user `76b5d1c7c9f24c2e981532b5d6097d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:04:59.804637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7415a6d4-a7cb-439c-9537-427cb3d340cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:59.891872 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1394/2790] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:04:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:04:59.904474 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:04:59.906779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:04:59.976186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47bb2-de97-4477-8a12-3eafd6109d49 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:04:59.980057 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:04:59.981029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:04:59.981029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:04:59.981244 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:00.004577 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:00.006262 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:00.018031 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1397/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:04:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:00.760168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c82b-35f2-4075-89ad-02b1ce94990d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:00.761275 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c82b-35f2-4075-89ad-02b1ce94990d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:00.761729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2c82b-35f2-4075-89ad-02b1ce94990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:00.791688 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d2f2c82b-35f2-4075-89ad-02b1ce94990d None None] MFA Rules not processed for user `842b7f1ea05f41d0b000f97ee9bf7196`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:00.793506 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2f2c82b-35f2-4075-89ad-02b1ce94990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:00.861320 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1395/2792] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:02.154999 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.156567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.262200 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.263152 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.263654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.285885 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a16c4-bd06-490b-af7a-a8661c76e804 service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.293374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.294079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.294364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.294641 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] PATH_INFO: `/v3/users/994d8a9fcee246aaa4b7357bdb3abf52` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.296137 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] RBAC: Authorizing `identity:delete_user(user_id=994d8a9fcee246aaa4b7357bdb3abf52)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.301881 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] MFA Rules not processed for user `8f8ef87496bb43d1bab7c5f6f101d6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:02.303613 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.304491 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.388805 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994d8a9fcee246aaa4b7357bdb3abf52'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:02.389269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:02.392527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:02.405603 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994d8a9fcee246aaa4b7357bdb3abf52'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:02.406368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994d8a9fcee246aaa4b7357bdb3abf52'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:02.418366 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1396/2793] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:05:02.418524 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994d8a9fcee246aaa4b7357bdb3abf52'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:02.423383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.424800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.439093 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-7d63dd4d-4d85-409b-891a-dfeb2aba549d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994d8a9fcee246aaa4b7357bdb3abf52'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:02.448175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12554119-95c3-4b04-ab6d-edbf842856ff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.448914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1398/2794] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:05:02 2022] DELETE /identity/v3/users/994d8a9fcee246aaa4b7357bdb3abf52 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:02.453274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.453630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.454331 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.478086 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:02.479674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.521552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.522171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.522451 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.522724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.523395 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.525638 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.575153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1397/2795] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:05:02 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:02.576262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.577860 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.581358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1399/2796] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:05:02.582732 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.587689 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:02.603535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17a9de-7cc8-4a45-bc05-4590ef0a296b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.654836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeedad2-eb31-4c89-a55e-7ce89ea162ff admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.659291 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.659650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.660137 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.661313 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.667064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.667749 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.668037 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.668310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.668940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.670871 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.700455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1400/2797] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:05:02 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:02.705417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.716305 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c170c-6c3c-464a-aa1e-68e7f0e50e3f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.720930 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.721527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.721808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.722079 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.722796 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.724978 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.735328 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1401/2798] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:02.739330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.745888 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.747422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.749436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab831264-ec7e-4da9-9331-58e1bf754489 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.754546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.755223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.755501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.755765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] PATH_INFO: `/v3/projects/67c67b8e9a924916a74636b162c5eeaf/users/cb3de078a9334ff9874cadcc51df222d/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.759671 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1398/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:02.772825 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c67b8e9a924916a74636b162c5eeaf, user_id=cb3de078a9334ff9874cadcc51df222d, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.776394 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.798728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1402/2800] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:02 2022] PUT /identity/v3/projects/67c67b8e9a924916a74636b162c5eeaf/users/cb3de078a9334ff9874cadcc51df222d/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:05:02.802811 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.812790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf861ae-7ce1-453d-bdac-5e4202230fb0 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.846464 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:02.898806 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.899516 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.899798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.900064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.900708 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.902771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.918609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1399/2801] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:02.922527 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:02.953448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f19b57-bf5d-4308-b371-f62aaada7621 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:02.960373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:02.961397 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:02.962133 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:02.962619 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] PATH_INFO: `/v3/projects/67c67b8e9a924916a74636b162c5eeaf/users/cb3de078a9334ff9874cadcc51df222d/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:02.965041 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67c67b8e9a924916a74636b162c5eeaf, user_id=cb3de078a9334ff9874cadcc51df222d, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:02.969637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814af7f1-ecfa-4c28-9a8e-c274b7b39a20 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:02.996126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1403/2802] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:02 2022] PUT /identity/v3/projects/67c67b8e9a924916a74636b162c5eeaf/users/cb3de078a9334ff9874cadcc51df222d/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:03.351312 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:03.354364 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:03.372642 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1259866-1b7b-431f-83e1-c6f3a4113238 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:03.507298 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:03.508102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:03.508452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:03.508831 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] PATH_INFO: `/v3/projects/0aa39f9ffa094aea95f5c5a9299880aa` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:03.510035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa39f9ffa094aea95f5c5a9299880aa)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:03.512885 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:03.591840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa39f9ffa094aea95f5c5a9299880aa', 'request_id': 'req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:03.602294 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa39f9ffa094aea95f5c5a9299880aa', 'request_id': 'req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:03.616976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa39f9ffa094aea95f5c5a9299880aa', 'request_id': 'req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:03.631840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] The token cache is being invalidate because project 0aa39f9ffa094aea95f5c5a9299880aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:05:03.632131 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:03.632875 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f426c2f-0c4b-41ac-b1c7-f2ebcfaa4f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:03.635035 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1400/2803] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:05:03 2022] DELETE /identity/v3/projects/0aa39f9ffa094aea95f5c5a9299880aa => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:03.642597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:03.643360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:03.643360 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:03.665192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:03.666960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:03.676714 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:03.787233 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1404/2804] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:03.796589 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:03.797749 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:03.798825 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:03.831270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:03.833913 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:03.849734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1401/2805] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:03.850923 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:03.861173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e74b1-3f0b-437a-866c-9901645f9ff4 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:03.868834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:03.868834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:03.868834 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:03.869209 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:03.869874 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80212) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:05:03.870083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:03.871771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:03.885571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1405/2806] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:05:03.890048 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:03.900899 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92743680-c438-4768-a9d5-4940ab3a0fcc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:03.905795 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:03.906669 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:03.907109 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:03.907474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:03.908269 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:03.913726 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:03.957184 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:03.962484 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1402/2807] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:05:03 2022] POST /identity/v3/projects => generated 361 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:03.969015 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109476c-0be9-489f-a961-ae17766a2fd2 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:04.062453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:04.063119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.063398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.063665 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] PATH_INFO: `/v3/users` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.064314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:04.068598 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:04.108406 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1406/2808] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:05:03 2022] POST /identity/v3/users => generated 281 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:05:04.111995 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:04.125127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063374ca-9fa9-4526-8a9f-58a2af1ddffd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:04.130189 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:04.130815 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.131352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.131477 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.132101 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:04.133724 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:04.146205 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1403/2809] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:04.150409 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:04.163081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0ab5-1de6-41d8-b5d5-3e1cb568a93d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:04.167842 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:04.168597 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.168879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.169149 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.169784 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:04.171571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9496e-474d-44dc-afa8-f12e75135baf admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:04.185803 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1407/2810] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:04.190210 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:04.202932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598f7a4-7591-4cb9-a2da-98b1da79d62f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:04.209299 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:04.210373 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.210821 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.211509 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] PATH_INFO: `/v3/projects/2c3a37b1eb54430b92e23e89a8cf5ef0/users/96d317c20b0449038f8905748ad52648/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.229965 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:04.234422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3a37b1eb54430b92e23e89a8cf5ef0, user_id=96d317c20b0449038f8905748ad52648, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:04.239262 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:04.261413 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1404/2811] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:04 2022] PUT /identity/v3/projects/2c3a37b1eb54430b92e23e89a8cf5ef0/users/96d317c20b0449038f8905748ad52648/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:05:04.273808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27235-41ea-4c76-b23b-21dbdd7515d1 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.274156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27235-41ea-4c76-b23b-21dbdd7515d1 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.274350 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b27235-41ea-4c76-b23b-21dbdd7515d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.317764 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-66b27235-41ea-4c76-b23b-21dbdd7515d1 None None] MFA Rules not processed for user `96d317c20b0449038f8905748ad52648`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:04.319661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-66b27235-41ea-4c76-b23b-21dbdd7515d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:04.435466 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1408/2812] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:05:04.449168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:04.453207 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:04.550629 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a37f4-64db-4609-b0ce-5b57558b114d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:04.555244 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3098494139cc41359d5c34b2a8428c6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:04.555970 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:04.556386 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:04.556848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:04.569633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:04.575541 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:04.588857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1405/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:09.343115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4a52-70f0-449a-873c-360577f78a06 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:09.343883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4a52-70f0-449a-873c-360577f78a06 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:09.346824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee4a52-70f0-449a-873c-360577f78a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:09.360058 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:09.389731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-14ee4a52-70f0-449a-873c-360577f78a06 None None] MFA Rules not processed for user `cb3de078a9334ff9874cadcc51df222d`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:09.391002 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-14ee4a52-70f0-449a-873c-360577f78a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:09.538782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1409/2814] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:09.559869 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:09.561332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:09.757178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee17eb-68f2-4812-bbd3-a4f665a1d3c7 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:09.770376 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:09.771501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:09.771501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:09.772156 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:09.774306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:09.862582 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:09.864898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:09.878063 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1406/2815] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:11.067741 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f14ae-4cc5-42af-adbf-06bca2dea393 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:11.068242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f14ae-4cc5-42af-adbf-06bca2dea393 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:11.068242 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f14ae-4cc5-42af-adbf-06bca2dea393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:11.093778 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-808f14ae-4cc5-42af-adbf-06bca2dea393 None None] MFA Rules not processed for user `8f8ef87496bb43d1bab7c5f6f101d6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:11.095450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-808f14ae-4cc5-42af-adbf-06bca2dea393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:11.106374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1410/2816] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:19.873568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:21.083783 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=35 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:51.743632 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:51.745076 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:51.828086 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c5f43-1ee3-48fd-9bb9-45406ab2b512 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:51.832538 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:51.833166 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:51.833444 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:51.833714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] PATH_INFO: `/v3/users/76b5d1c7c9f24c2e981532b5d6097d66` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:51.834836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b5d1c7c9f24c2e981532b5d6097d66)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:51.836799 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:51.890771 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b5d1c7c9f24c2e981532b5d6097d66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:51.901706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b5d1c7c9f24c2e981532b5d6097d66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:51.902559 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b5d1c7c9f24c2e981532b5d6097d66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:51.917008 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b5d1c7c9f24c2e981532b5d6097d66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:51.925160 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b5d1c7c9f24c2e981532b5d6097d66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:51.932721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1407/2817] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:05:51 2022] DELETE /identity/v3/users/76b5d1c7c9f24c2e981532b5d6097d66 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:51.937191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:51.937727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:51.937727 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:51.957984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:51.959103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:52.020437 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1411/2818] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:52.030301 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:52.031721 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:52.093643 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b785b9c-aae3-4dc4-bffc-e158ea705732 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:52.097503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:52.098802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.098802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.098802 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.111975 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:52.113912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d2d8-1452-436f-acfe-b88bef9ef572 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:52.127958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1408/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:52.573984 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:52.575782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:52.589585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed424ce-7788-4a8a-a661-ee37bef46681 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:52.595374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:52.596173 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.596453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.596719 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] PATH_INFO: `/v3/projects/3fdb6c1641c74f97a10f57644b87197b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.597832 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fdb6c1641c74f97a10f57644b87197b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:52.600020 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:52.667998 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fdb6c1641c74f97a10f57644b87197b', 'request_id': 'req-46303b75-4f14-4324-aa24-d6712c84badd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:52.674506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fdb6c1641c74f97a10f57644b87197b', 'request_id': 'req-46303b75-4f14-4324-aa24-d6712c84badd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:52.682248 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fdb6c1641c74f97a10f57644b87197b', 'request_id': 'req-46303b75-4f14-4324-aa24-d6712c84badd'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:52.697124 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] The token cache is being invalidate because project 3fdb6c1641c74f97a10f57644b87197b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:05:52.697408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:52.698036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-46303b75-4f14-4324-aa24-d6712c84badd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:52.699557 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1412/2820] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:05:52 2022] DELETE /identity/v3/projects/3fdb6c1641c74f97a10f57644b87197b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:52.704729 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.705021 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.705308 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.738871 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:52.740099 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:52.758927 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:52.816473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1409/2821] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:52.828073 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.828405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.828795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.883901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:52.887236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:52.908537 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1413/2822] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:52.912866 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:52.923830 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379a82d-3598-4d55-b385-2ed8668be838 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:52.930133 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:52.930497 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.931022 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.931500 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.932305 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:05:52.932758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:52.935347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:52.952485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1410/2823] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 18:05:52.954243 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:52.966738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f70984-54f1-4127-b4e1-104de9e3167c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:52.972366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:52.972991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:52.973267 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:52.973534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:52.974160 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:52.976210 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.012946 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1414/2824] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:05:52 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:53.017009 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:53.028491 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e3f5d-1dcf-4629-a945-718b6c1aa5ad admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:53.091792 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:53.092396 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.092676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.092942 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.093570 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:53.095519 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.117268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1411/2825] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:05:53 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:05:53.121762 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:53.132212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9a79-c227-40f5-8f83-c2881fc74d9d admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:53.137647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:53.138541 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.138944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.139253 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.139927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:53.141706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.152614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1415/2826] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:53.156947 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:53.167120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667971e0-67db-4617-9341-64db922d11a4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:53.172667 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:53.173007 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.173285 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.173553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.174502 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:53.175855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa3903-dcba-4fe5-991b-bcd6948c5eeb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.186594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1412/2827] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:05:53.190787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:53.201652 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53b051-fb83-4ff6-b210-8161952fa872 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:53.206368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:53.207141 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.207417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.207780 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] PATH_INFO: `/v3/projects/4821ddb8e299400eba9adc4bc53ebe6b/users/8bf28be46c9c4d9c9f5a7c5837545fa4/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.224036 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4821ddb8e299400eba9adc4bc53ebe6b, user_id=8bf28be46c9c4d9c9f5a7c5837545fa4, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:53.226685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.239545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1416/2828] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:53 2022] PUT /identity/v3/projects/4821ddb8e299400eba9adc4bc53ebe6b/users/8bf28be46c9c4d9c9f5a7c5837545fa4/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:53.244682 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.244969 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.245236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.264781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:53.265981 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:53.326018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1413/2829] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:53.335769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:53.340022 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:53.361374 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:53.398702 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2ac2f-b451-42e1-9b53-bb7742920fd6 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:53.402322 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:53.402920 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:53.403231 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:53.403505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:53.412339 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:53.413769 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba9a84-5d9b-4ba1-88c8-3d052d399679 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:53.423631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1417/2830] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:56.846952 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:56.848716 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:56.861641 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc86bd-416a-413c-b9cf-450f80261f0e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:56.867715 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:56.868039 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:56.868317 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:56.868587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] PATH_INFO: `/v3/users/96d317c20b0449038f8905748ad52648` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:56.870027 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d317c20b0449038f8905748ad52648)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:56.871845 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:56.897964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:56.948092 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d317c20b0449038f8905748ad52648'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:56.958596 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d317c20b0449038f8905748ad52648'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:56.960901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d317c20b0449038f8905748ad52648'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:56.976706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d317c20b0449038f8905748ad52648'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:56.992511 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d317c20b0449038f8905748ad52648'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:57.009603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1414/2831] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:05:56 2022] DELETE /identity/v3/users/96d317c20b0449038f8905748ad52648 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:05:57.016430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:57.017106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:57.017106 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:57.061670 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:57.063336 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:57.186795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1418/2832] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:57.199674 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:57.201504 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:57.294164 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45198a-6def-41c2-b7db-96a2ac74a1a8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:57.299909 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:57.300531 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:57.300809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:57.301221 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:57.312263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:57.313756 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8894e5-7d78-4d2a-bfed-3aa026132ff2 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:57.328978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1415/2833] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:05:57.895534 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:57.897882 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:57.917369 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6a1f9-3ec1-4682-8041-f422dcb6731b None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:57.936571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:57.937508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:57.937838 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:57.938146 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] PATH_INFO: `/v3/projects/2c3a37b1eb54430b92e23e89a8cf5ef0` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:57.939744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3a37b1eb54430b92e23e89a8cf5ef0)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:57.941487 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.036195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3a37b1eb54430b92e23e89a8cf5ef0', 'request_id': 'req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:58.046212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3a37b1eb54430b92e23e89a8cf5ef0', 'request_id': 'req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:58.062046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3a37b1eb54430b92e23e89a8cf5ef0', 'request_id': 'req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:58.089109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] The token cache is being invalidate because project 2c3a37b1eb54430b92e23e89a8cf5ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:05:58.090169 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:58.091131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:05:58.094045 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1419/2834] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:05:57 2022] DELETE /identity/v3/projects/2c3a37b1eb54430b92e23e89a8cf5ef0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:58.109936 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.111603 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.112126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.152416 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] MFA Rules not processed for user `8bf28be46c9c4d9c9f5a7c5837545fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:05:58.154090 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:58.234754 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1416/2835] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:05:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:05:58.240613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.242110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:05:58.308763 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:58.332435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2a5a-59c0-450e-b2fa-bed8ad22c1b5 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.339941 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.340957 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.341307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.341654 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.342371 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.344629 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.389612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1420/2836] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:05:58 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:58.393734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.405472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d73de-3df6-4c33-ab87-33a83d3a6450 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.444391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:05:58.473950 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.474707 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.475014 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.475585 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.476266 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.478798 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.501388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1417/2837] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:05:58 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:05:58.505747 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.516251 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a1131-2457-44c4-9e92-cfca3a836f99 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.520975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.521604 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.522085 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.522362 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.523048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.524687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.535274 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1421/2838] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:58.539434 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.550828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c74d8-4cdf-42c4-bbfe-75929ae08f68 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.556240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.556901 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.557209 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.557480 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] PATH_INFO: `/v3/projects/4c240e3a80e2484695d991b45538ff4c/users/5069d76b8fbe49aca57da73f4495a8cc/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.573903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c240e3a80e2484695d991b45538ff4c, user_id=5069d76b8fbe49aca57da73f4495a8cc, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.576824 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.591764 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1418/2839] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:58 2022] PUT /identity/v3/projects/4c240e3a80e2484695d991b45538ff4c/users/5069d76b8fbe49aca57da73f4495a8cc/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:05:58.595752 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.606080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928983c-d50b-4a3e-8054-4502c9ef3fce admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.662216 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.662850 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.663163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.663436 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.664103 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.665724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91c739-f336-4c68-afd3-1f9ec04b3a9a admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.675361 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1422/2840] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:05:58.679874 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:05:58.690687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb9137-60f1-458b-9f89-68c7e87aa125 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:05:58.695920 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:05:58.697108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:05:58.697380 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:05:58.697645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] PATH_INFO: `/v3/projects/4c240e3a80e2484695d991b45538ff4c/users/5069d76b8fbe49aca57da73f4495a8cc/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:05:58.699558 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c240e3a80e2484695d991b45538ff4c, user_id=5069d76b8fbe49aca57da73f4495a8cc, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:05:58.702212 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:05:58.712501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1419/2841] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:05:58 2022] PUT /identity/v3/projects/4c240e3a80e2484695d991b45538ff4c/users/5069d76b8fbe49aca57da73f4495a8cc/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:06:03.584606 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:03.585091 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:03.585195 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:03.679162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] MFA Rules not processed for user `5069d76b8fbe49aca57da73f4495a8cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:03.680706 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:03.779385 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1423/2842] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:03.795903 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:03.797706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:03.917820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44626f-1bc4-4957-929b-b5db8d83b813 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:03.924930 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:03.925884 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:03.926242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:03.926676 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:04.019040 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:04.020828 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:04.033256 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1420/2843] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:05.973569 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:05.975017 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:06.021204 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:06.052172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6889d-08cb-4e5f-b5b9-b5042af322fb None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:06.056500 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:06.057167 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:06.057678 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:06.057975 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:06.064075 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:06.080268 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:06.081721 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:06.093724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1424/2844] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:08.490994 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:08.501127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd950-1af7-4f41-a68a-b1e71ab8a388 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:08.505117 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:08.505730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:08.506054 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:08.506344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:08.508153 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:08.523047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:08.524486 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:08.535472 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1421/2845] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:10.063575 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:10.072698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc7d67-86e2-4dac-aa35-40a23e47b35e service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:10.076470 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:10.077068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:10.077421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:10.077611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:10.079435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:10.144929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:10.146453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:10.159365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1425/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:13.844267 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:13.845713 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:13.856383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:13.916912 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755f3fc-57a3-4238-af78-9b5b134df259 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:13.921557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:13.922179 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:13.922457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:13.922730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] PATH_INFO: `/v3/users/842b7f1ea05f41d0b000f97ee9bf7196` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:13.923876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842b7f1ea05f41d0b000f97ee9bf7196)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:13.925790 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:13.970955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842b7f1ea05f41d0b000f97ee9bf7196'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:13.980955 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842b7f1ea05f41d0b000f97ee9bf7196'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:13.981575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842b7f1ea05f41d0b000f97ee9bf7196'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:13.994321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842b7f1ea05f41d0b000f97ee9bf7196'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.001176 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e9daa171-0639-470c-b6df-4e03858bc1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842b7f1ea05f41d0b000f97ee9bf7196'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.007306 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1422/2847] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:06:13 2022] DELETE /identity/v3/users/842b7f1ea05f41d0b000f97ee9bf7196 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:06:14.212192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.224712 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd41081-1916-4af2-b461-797accb2e8fc service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.295616 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.296191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.296450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.296724 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] PATH_INFO: `/v3/projects/fae6073847f2416da72f6c9276da44cf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.297953 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae6073847f2416da72f6c9276da44cf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.300006 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.354673 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae6073847f2416da72f6c9276da44cf', 'request_id': 'req-65eb3740-048c-4ccc-adae-1cd8428dc14d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.360927 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae6073847f2416da72f6c9276da44cf', 'request_id': 'req-65eb3740-048c-4ccc-adae-1cd8428dc14d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.367349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae6073847f2416da72f6c9276da44cf', 'request_id': 'req-65eb3740-048c-4ccc-adae-1cd8428dc14d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.377408 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:14.381914 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] The token cache is being invalidate because project fae6073847f2416da72f6c9276da44cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:06:14.382368 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.383508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-65eb3740-048c-4ccc-adae-1cd8428dc14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:06:14.385203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1426/2848] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:06:14 2022] DELETE /identity/v3/projects/fae6073847f2416da72f6c9276da44cf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:06:14.390382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.390837 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.391144 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.411794 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:14.412984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:14.471979 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1423/2849] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:14.480314 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.480627 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.480907 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.500889 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:14.502492 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:14.510355 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1427/2850] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:14.514616 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.524932 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f1d54-020c-4b51-9120-b42f6bc5d150 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.529365 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.529954 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.530230 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.530501 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.531169 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 18:06:14.531474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.533172 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.546728 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1424/2851] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 9 18:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 18:06:14.550812 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.562049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0a533-11fb-4f00-acdd-15234e1c2f18 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.568909 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.569505 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.569779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.570048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.570667 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.572613 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.601183 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1428/2852] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:06:14 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:06:14.606018 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.616064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf123a-7ae1-47f2-b4a8-62aacc5bde13 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.627588 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:14.676695 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.677302 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.677575 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.677842 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.678474 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.680449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.702303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1425/2853] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:06:14 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:06:14.706738 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.717497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be8398-749d-4e09-8948-a8a694f56698 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.722471 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.723100 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.723384 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.723657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.724423 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.726271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.736802 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1429/2854] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:06:14.741069 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.751332 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffee40-1593-427c-b12e-ad0152fc13d8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.756243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.756840 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.757118 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.757388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.758083 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.760568 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaf8f2-e9f7-4b94-9788-e81423e24344 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.771452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1426/2855] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:06:14.776214 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.787808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8b1ca-a1a5-4138-86d4-6782e0b4b7c0 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.793546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.794199 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.794484 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.794755 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] PATH_INFO: `/v3/projects/1b9d999e96684373b7fe94b07d007eee/users/83be648a00b24d7082a01c56f4317069/roles/b424450c766b4e32b0c1dbb8d5088e4d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.809572 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9d999e96684373b7fe94b07d007eee, user_id=83be648a00b24d7082a01c56f4317069, role_id=b424450c766b4e32b0c1dbb8d5088e4d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:14.812186 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:14.825971 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1430/2856] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:06:14 2022] PUT /identity/v3/projects/1b9d999e96684373b7fe94b07d007eee/users/83be648a00b24d7082a01c56f4317069/roles/b424450c766b4e32b0c1dbb8d5088e4d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:06:14.831120 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5998ae-240e-4453-a4ff-4a423cf0b418 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.831449 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5998ae-240e-4453-a4ff-4a423cf0b418 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.831722 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5998ae-240e-4453-a4ff-4a423cf0b418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:14.852264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fb5998ae-240e-4453-a4ff-4a423cf0b418 None None] MFA Rules not processed for user `654dd146616b4cbd8975ee15d286d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:14.853473 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fb5998ae-240e-4453-a4ff-4a423cf0b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:14.916113 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1427/2857] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:14.926413 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:14.928153 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:14.991370 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8f5b-78ea-4902-a977-af565b6ebc9b admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:14.994944 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:14.995565 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:14.995840 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:14.996108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:15.005292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:15.006720 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:15.019860 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1431/2858] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:18.913714 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:18.914217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:18.914316 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:18.943857 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] MFA Rules not processed for user `83be648a00b24d7082a01c56f4317069`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:18.945072 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:19.014321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1428/2859] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:19.018962 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.028892 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ec875-d0c0-4dde-87aa-6e0e5fb1d769 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.033551 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.034151 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.034435 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.034711 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] PATH_INFO: `/v3/projects` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.035402 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.037308 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.068319 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1432/2860] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 9 18:06:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 18:06:19.074469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.085748 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a178f-65cd-4db7-a564-da05a5eb3a89 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.108059 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:19.148260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.149412 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.149693 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.149964 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.150607 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.152594 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.175423 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1429/2861] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 9 18:06:19 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 18:06:19.179261 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.193246 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914fc71-6d67-4e98-918b-5aa6f693674c admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.198098 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.198699 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.198997 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.199284 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.199926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.201576 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.213453 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1433/2862] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 18:06:19.217623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.228457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051916f3-c452-4cd2-b5a2-b409a467999d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.233188 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.233839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.234122 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.234397 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] PATH_INFO: `/v3/projects/fa1eedc5284442a58b0374535cc62fdf/users/cf31d17b84d640eda20881865b53b6d3/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.251125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1eedc5284442a58b0374535cc62fdf, user_id=cf31d17b84d640eda20881865b53b6d3, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.254129 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.270347 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1430/2863] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:06:19 2022] PUT /identity/v3/projects/fa1eedc5284442a58b0374535cc62fdf/users/cf31d17b84d640eda20881865b53b6d3/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:06:19.274486 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.285187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35146c-6fb4-45bc-8847-ad0ef7e0f894 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.343282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.343905 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.344212 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.344481 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.345112 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.346691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59dfe0-c2f7-4850-8714-7148b39236b1 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.357197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1434/2864] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 9 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 18:06:19.361681 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:19.373264 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613ae71-49ca-4225-85cc-0a63a30f6a16 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:19.381773 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:19.382430 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:19.382712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:19.383001 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] PATH_INFO: `/v3/projects/fa1eedc5284442a58b0374535cc62fdf/users/cf31d17b84d640eda20881865b53b6d3/roles/7c95125003c7415cb3d20dd52bbf8e1c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:19.385343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1eedc5284442a58b0374535cc62fdf, user_id=cf31d17b84d640eda20881865b53b6d3, role_id=7c95125003c7415cb3d20dd52bbf8e1c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:19.387817 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2515783-0c98-4340-9064-3f1609ed002d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:19.403344 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1431/2865] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 9 18:06:19 2022] PUT /identity/v3/projects/fa1eedc5284442a58b0374535cc62fdf/users/cf31d17b84d640eda20881865b53b6d3/roles/7c95125003c7415cb3d20dd52bbf8e1c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:06:21.724612 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:21.725172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:21.725172 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:21.736399 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:21.754307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] MFA Rules not processed for user `8bf28be46c9c4d9c9f5a7c5837545fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:21.755564 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:21.822824 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1435/2866] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:24.257263 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499b751-1f7c-4c05-9b4c-531a52c43f54 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:24.258440 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499b751-1f7c-4c05-9b4c-531a52c43f54 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:24.258712 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499b751-1f7c-4c05-9b4c-531a52c43f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:24.291402 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7499b751-1f7c-4c05-9b4c-531a52c43f54 None None] MFA Rules not processed for user `cf31d17b84d640eda20881865b53b6d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:24.292747 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7499b751-1f7c-4c05-9b4c-531a52c43f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:24.388804 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1432/2867] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:24.399468 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:24.401064 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:24.469883 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1b6a9-155d-4743-884d-736694de2dd7 None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:24.474269 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b46658fdbcde40498e87d37d79609c83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:24.475110 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:24.475456 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:24.475792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:24.547237 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:24.548698 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:24.559792 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1436/2868] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:24.694060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a1ce-7f37-47dd-8cae-9f1cb8dbe37d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:24.694485 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a1ce-7f37-47dd-8cae-9f1cb8dbe37d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:24.694848 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a1ce-7f37-47dd-8cae-9f1cb8dbe37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:24.735846 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f369a1ce-7f37-47dd-8cae-9f1cb8dbe37d None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:24.737100 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f369a1ce-7f37-47dd-8cae-9f1cb8dbe37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:24.794190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:24.803807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1433/2869] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:24.852295 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:24.853693 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:24.918782 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=100 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:24.923280 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbe15f-0086-4e79-9960-b935d815a0ee service glance] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:24.928417 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:24.929013 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:24.929292 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:24.929567 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:24.938388 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:24.939951 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a31f1-dee6-4b4b-8320-eefddc6fa908 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:24.948365 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1437/2870] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:06:34.459218 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007925e0-c509-4201-aef2-97306c474b87 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:34.459832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007925e0-c509-4201-aef2-97306c474b87 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:34.459832 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007925e0-c509-4201-aef2-97306c474b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:34.479567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-007925e0-c509-4201-aef2-97306c474b87 None None] MFA Rules not processed for user `83be648a00b24d7082a01c56f4317069`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:34.480774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-007925e0-c509-4201-aef2-97306c474b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:34.487787 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1434/2871] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:06:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:06:34.944643 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=4 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:36.388571 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:36.389170 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:36.389382 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:36.450010 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:06:36.451271 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:36.475156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1438/2872] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 9 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:06:36.484409 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007925e0-c509-4201-aef2-97306c474b87 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:36.510672 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007925e0-c509-4201-aef2-97306c474b87 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:36.516839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:36.517047 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:36.517287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:36.517550 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:36.527945 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:36.529149 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:36.540073 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1435/2873] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:06:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 18:06:45.712282 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:45.714016 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:45.803448 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1607f-70bd-418c-8f1c-3ebd1ba0f85d None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:45.808898 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:45.809508 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:45.809787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:45.810060 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:45.811820 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:45.824637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:45.826206 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:45.837428 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1439/2874] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:46.535727 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=14 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:06:47.786650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:47.788321 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:47.864389 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253b5ca-66e6-4d04-b998-e04b1797c4fe service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:47.869029 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:47.869741 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:47.870190 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:47.870493 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:47.872576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:47.885292 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:47.887201 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:47.897706 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1436/2875] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:49.359156 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:49.369887 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d630a5-9ae1-4d28-b3fc-b655bbd3247b service nova] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:49.373529 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:49.374126 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:49.374405 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:49.374675 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:49.376454 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:49.439443 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:49.440866 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:49.450273 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1440/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:51.129548 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:06:51.131044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:51.147377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87723492-6300-4bc5-9ed6-5cf77d3b5a77 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:06:51.154078 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:06:51.154687 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:06:51.155127 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:06:51.155407 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:06:51.157167 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:06:51.170439 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:06:51.172123 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ebfd4-a2cd-4934-b039-b463de57aa12 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:06:51.183030 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1437/2877] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:06:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:06:59.446025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=82 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:01.178374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:07:05.218373 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:07:05.227175 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbefae-080d-4f82-8d82-ff19f961811c service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:07:05.231680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:07:05.231888 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:07:05.232162 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:07:05.232429 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:07:05.241223 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:07:05.242637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:07:05.251862 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1441/2878] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:07:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:07:15.247868 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:26.710547 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:26.711097 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:26.711227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:26.730242 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] MFA Rules not processed for user `3a73ecd2399f434ba5999aa5be2f7fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:26.731499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:26.738655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1438/2879] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 9 18:08:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:08:26.779893 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:26.790989 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f79c2a-77ef-4833-bc25-b38606042c23 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:26.796046 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae392d38ade4665b2385bae93b3e688', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:26.796801 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:26.797270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:26.797631 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:26.806546 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:26.808143 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:26.816549 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1442/2880] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 9 18:08:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 18:08:27.038601 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:27.052503 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3e368-46db-4cd5-b99d-c247a052b7ad None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:27.062876 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:27.064499 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:27.065415 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:27.065905 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:27.073289 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.086924 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:27.088656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:27.110775 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1439/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:27.708917 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:27.735740 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db3185-4543-495a-acb0-1baa56a9fa95 service swift] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:27.739615 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:27.740217 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:27.740496 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:27.740766 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:27.743332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:27.754194 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:27.755661 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:27.765021 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1443/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:28.902469 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:28.914476 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7441bf-911c-4aa7-ab01-f080893fba7c service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:28.919203 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:28.920098 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:28.920377 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:28.920647 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:28.922536 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:28.940403 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:28.941851 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:28.959997 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1440/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:29.083991 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:29.086310 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.184241 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041df4-c14e-410a-90bc-c7ae24b647e0 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:29.189895 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:29.190512 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:29.190808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:29.191108 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:29.192847 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:29.205545 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:29.207192 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:29.217548 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1444/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:08:33.810757 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:33.877623 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923a78a-eeed-4393-a64a-ac59089caf64 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:33.882240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:33.882867 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:33.883181 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:33.883459 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] PATH_INFO: `/v3/users/83be648a00b24d7082a01c56f4317069` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:33.884590 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83be648a00b24d7082a01c56f4317069)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:33.886513 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:33.934574 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83be648a00b24d7082a01c56f4317069'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:33.943705 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83be648a00b24d7082a01c56f4317069'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:33.944354 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83be648a00b24d7082a01c56f4317069'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:33.957178 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83be648a00b24d7082a01c56f4317069'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:33.964910 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83be648a00b24d7082a01c56f4317069'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:33.971651 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1441/2885] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:08:33 2022] DELETE /identity/v3/users/83be648a00b24d7082a01c56f4317069 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:08:34.395956 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:34.409765 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16c795-059f-4ee2-827d-fe9753c9e3e3 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:34.476879 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:34.477530 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:34.477808 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:34.478080 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] PATH_INFO: `/v3/projects/1b9d999e96684373b7fe94b07d007eee` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:34.479219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9d999e96684373b7fe94b07d007eee)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:34.481115 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:34.525591 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9d999e96684373b7fe94b07d007eee', 'request_id': 'req-317e9b32-3e79-419e-90ab-20d9260a5390'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.530934 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9d999e96684373b7fe94b07d007eee', 'request_id': 'req-317e9b32-3e79-419e-90ab-20d9260a5390'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.537680 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9d999e96684373b7fe94b07d007eee', 'request_id': 'req-317e9b32-3e79-419e-90ab-20d9260a5390'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.540185 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:34.552109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] The token cache is being invalidate because project 1b9d999e96684373b7fe94b07d007eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:08:34.552520 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.553342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.555001 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1445/2886] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:08:34 2022] DELETE /identity/v3/projects/1b9d999e96684373b7fe94b07d007eee => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:08:34.559091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:34.560633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:34.573431 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0276070-6df1-44f4-9c3f-a45de02a40c8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:34.617863 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:34.632552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:34.633180 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:34.633460 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:34.633735 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] PATH_INFO: `/v3/users/cf31d17b84d640eda20881865b53b6d3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:34.634855 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf31d17b84d640eda20881865b53b6d3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:34.636807 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:34.681555 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf31d17b84d640eda20881865b53b6d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.689031 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf31d17b84d640eda20881865b53b6d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.689736 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf31d17b84d640eda20881865b53b6d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.701923 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf31d17b84d640eda20881865b53b6d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.709743 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0096379a-2e2c-4145-8287-8b95dfd96323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf31d17b84d640eda20881865b53b6d3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:34.715553 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1442/2887] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:08:34 2022] DELETE /identity/v3/users/cf31d17b84d640eda20881865b53b6d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:08:34.895422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:34.897007 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:34.911691 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e9b32-3e79-419e-90ab-20d9260a5390 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:34.970926 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:34.971509 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:34.971799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:34.972061 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] PATH_INFO: `/v3/projects/fa1eedc5284442a58b0374535cc62fdf` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:34.973203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1eedc5284442a58b0374535cc62fdf)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:34.975132 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:35.021786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1eedc5284442a58b0374535cc62fdf', 'request_id': 'req-f2719619-a669-4e06-99e2-11daaec03f00'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:35.027264 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1eedc5284442a58b0374535cc62fdf', 'request_id': 'req-f2719619-a669-4e06-99e2-11daaec03f00'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:35.033497 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1eedc5284442a58b0374535cc62fdf', 'request_id': 'req-f2719619-a669-4e06-99e2-11daaec03f00'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:35.045715 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] The token cache is being invalidate because project fa1eedc5284442a58b0374535cc62fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:08:35.046029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:35.046658 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:08:35.048197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1446/2888] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:08:34 2022] DELETE /identity/v3/projects/fa1eedc5284442a58b0374535cc62fdf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:08:44.712193 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:45.042501 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:08:52.403060 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:52.403587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:52.403587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:52.423737 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] MFA Rules not processed for user `8bf28be46c9c4d9c9f5a7c5837545fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:08:52.424958 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:52.482450 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1443/2889] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:08:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 18:08:52.795929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:08:52.797342 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:52.863935 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2719619-a669-4e06-99e2-11daaec03f00 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:08:52.867573 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:08:52.868165 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:08:52.868440 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:08:52.868708 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:08:52.870450 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:08:52.880052 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:08:52.881455 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:08:52.889421 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1447/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:08:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:02.475789 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=25 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:02.885744 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=26 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:09:20.144171 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:09:20.145587 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:20.215214 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41bd10-33a5-4dad-a255-bc390d5a421e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:09:20.220125 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:09:20.220723 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:09:20.221000 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:09:20.221268 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:09:20.223093 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:09:20.235235 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:09:20.236813 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:09:20.250656 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1444/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:09:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:09:30.244978 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=32 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:08.831130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:08.832515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:08.843787 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a87459-d72b-4043-b673-e75dfbeda2bf service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:08.847535 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:08.848131 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:08.848441 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:08.849265 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:08.850637 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:08.914515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:08.916490 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:08.926358 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1448/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:11.664774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:11.674637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc5a56-64c8-4e47-851c-2edfe2ed0369 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:11.678050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:11.678628 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:11.678902 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:11.679197 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:11.681567 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:11.742296 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:11.744345 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:11.754080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1445/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:14.874109 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:14.889286 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f5d503-b925-49ab-9443-7afcb25a6dfb service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:14.895881 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:14.896608 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:14.896990 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:14.897366 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:14.899589 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:14.912939 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:14.914915 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:14.933025 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1449/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:10:18.450597 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:18.463734 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e26af-ce4a-474b-9952-0c9a233aaa63 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:18.469615 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:18.470452 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:18.470758 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:18.471064 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:18.472820 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:18.488319 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:18.489966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:18.506891 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1446/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:10:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:10:18.520138 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:18.521674 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:18.536815 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c88e54-fc40-4e4f-b630-c44485c0a9f0 service placement] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:18.542283 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:18.542901 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:18.543228 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:18.543539 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] PATH_INFO: `/v3/users/8f8ef87496bb43d1bab7c5f6f101d6aa` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:18.544685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8ef87496bb43d1bab7c5f6f101d6aa)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:18.546602 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:18.594710 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8ef87496bb43d1bab7c5f6f101d6aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.601620 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8ef87496bb43d1bab7c5f6f101d6aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.602356 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8ef87496bb43d1bab7c5f6f101d6aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.614647 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8ef87496bb43d1bab7c5f6f101d6aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.628687 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8ef87496bb43d1bab7c5f6f101d6aa'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.634390 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1450/2896] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:10:18 2022] DELETE /identity/v3/users/8f8ef87496bb43d1bab7c5f6f101d6aa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:10:18.805751 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:18.817175 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913741b-c079-46fd-8baa-cdf803caa6aa service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:18.873935 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:18.874552 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:18.874827 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:18.875126 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] PATH_INFO: `/v3/projects/d6b4958df154416a820335d272cec49d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:18.876227 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b4958df154416a820335d272cec49d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:18.878375 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:18.926260 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b4958df154416a820335d272cec49d', 'request_id': 'req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.933598 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b4958df154416a820335d272cec49d', 'request_id': 'req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.940277 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b4958df154416a820335d272cec49d', 'request_id': 'req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.950774 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] The token cache is being invalidate because project d6b4958df154416a820335d272cec49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:10:18.951089 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.951740 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:18.953217 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1447/2897] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:10:18 2022] DELETE /identity/v3/projects/d6b4958df154416a820335d272cec49d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:10:18.957236 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:18.958822 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:18.972728 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc03544-b6af-4ebd-8912-ecb6e56f4e59 admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:19.023119 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:19.030485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:19.031288 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:19.031929 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:19.032377 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] PATH_INFO: `/v3/users/cb3de078a9334ff9874cadcc51df222d` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:19.033869 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3de078a9334ff9874cadcc51df222d)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:19.036257 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:19.084130 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3de078a9334ff9874cadcc51df222d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.091779 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3de078a9334ff9874cadcc51df222d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.092445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3de078a9334ff9874cadcc51df222d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.099626 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3de078a9334ff9874cadcc51df222d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.111346 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-018d3189-ff0a-4f7e-a64b-6652c6d9df4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3de078a9334ff9874cadcc51df222d'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.117084 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1451/2898] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:10:18 2022] DELETE /identity/v3/users/cb3de078a9334ff9874cadcc51df222d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:10:19.527352 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:10:19.528951 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:19.546956 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc297e3-7aef-46a4-b8c2-c26a01ff1d23 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:10:19.554858 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:19.609079 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:10:19.609711 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:19.609988 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:19.610259 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] PATH_INFO: `/v3/projects/67c67b8e9a924916a74636b162c5eeaf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:19.611406 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67c67b8e9a924916a74636b162c5eeaf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:10:19.613315 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:10:19.660622 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67c67b8e9a924916a74636b162c5eeaf', 'request_id': 'req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.668610 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67c67b8e9a924916a74636b162c5eeaf', 'request_id': 'req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.674839 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67c67b8e9a924916a74636b162c5eeaf', 'request_id': 'req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.685698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] The token cache is being invalidate because project 67c67b8e9a924916a74636b162c5eeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:10:19.685984 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.686609 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:10:19.688107 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1448/2899] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:10:19 2022] DELETE /identity/v3/projects/67c67b8e9a924916a74636b162c5eeaf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:10:29.113657 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:29.681892 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:10:48.201134 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] REQUEST_METHOD: `POST` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:10:48.201681 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:10:48.201799 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:10:48.220191 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.auth.core [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] MFA Rules not processed for user `8bf28be46c9c4d9c9f5a7c5837545fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80212) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 18:10:48.221379 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:10:48.276771 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1452/2900] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 9 18:10:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 18:10:58.270158 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=25 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:11:04.163635 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:11:04.165038 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:04.238258 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f168e53-fac7-4a89-a6f8-cfde86a35cb5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:11:04.243783 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:11:04.244388 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:11:04.244668 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:11:04.244944 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:11:04.246680 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:11:04.257080 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:11:04.258645 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:11:04.276165 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1449/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:11:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:11:14.271966 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=34 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:12:18.585422 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:12:18.586921 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:18.647398 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4f2e8-261f-46a8-90db-2f5e4700891c None None] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:12:18.651187 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:12:18.651795 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:12:18.652068 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:12:18.652330 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:12:18.654029 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:18.665203 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:12:18.666614 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:12:18.675655 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1453/2902] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:12:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:12:28.671270 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=26 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:12:46.257303 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:12:46.269579 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b467812-819b-4ee8-8c61-4377200c53d3 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:12:46.277557 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a136641c41464f1f969fe6c16dfe3501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:12:46.278418 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:12:46.279170 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:12:46.279457 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:12:46.281654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:46.294363 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:12:46.297142 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:12:46.308128 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1450/2903] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:12:46.456432 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:12:46.458083 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:46.470048 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ab607-aac2-4374-88ef-06a0e8c7484e service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:12:46.473685 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:12:46.474276 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:12:46.474555 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:12:46.474819 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:12:46.476586 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:46.491485 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:12:46.492940 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:12:46.507349 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1454/2904] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:12:47.695437 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:12:47.704781 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d6aba-d79d-4a60-a33a-5e48e114411a service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:12:47.709745 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:12:47.710102 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:12:47.710383 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:12:47.710654 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:12:47.712422 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:47.728637 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:12:47.730108 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:12:47.741576 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1451/2905] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:12:56.499272 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=6 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:12:57.736445 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=12 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:12:59.117587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:12:59.126731 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569842df-370d-4597-a2ba-de8f64ed5194 service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:12:59.130880 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '768dabebad264236b8f43dbd1c85a964', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:12:59.131510 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] REQUEST_METHOD: `GET` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:12:59.131786 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:12:59.132051 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:12:59.133772 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:12:59.200777 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:12:59.202263 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:12:59.212430 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1455/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 18:13:05.741270 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:05.743055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:05.757050 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40e8c-50dd-4f34-aaa8-1a2e81562708 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:05.761626 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:05.762187 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:05.762487 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:05.762760 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] PATH_INFO: `/v3/users/8bf28be46c9c4d9c9f5a7c5837545fa4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:05.763896 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf28be46c9c4d9c9f5a7c5837545fa4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:05.765809 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:05.813374 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf28be46c9c4d9c9f5a7c5837545fa4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:05.821044 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf28be46c9c4d9c9f5a7c5837545fa4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:05.821829 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf28be46c9c4d9c9f5a7c5837545fa4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:05.834461 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf28be46c9c4d9c9f5a7c5837545fa4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:05.841240 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf28be46c9c4d9c9f5a7c5837545fa4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:05.847094 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1452/2907] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:13:05 2022] DELETE /identity/v3/users/8bf28be46c9c4d9c9f5a7c5837545fa4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 18:13:06.027049 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:06.039042 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b02ff8-3552-4113-ab10-ffc6f240e54b service neutron] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:06.102332 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:06.102960 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:06.103307 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:06.103585 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] PATH_INFO: `/v3/projects/4821ddb8e299400eba9adc4bc53ebe6b` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:06.104716 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4821ddb8e299400eba9adc4bc53ebe6b)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:06.106611 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:06.157081 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4821ddb8e299400eba9adc4bc53ebe6b', 'request_id': 'req-d495ebb7-3fa5-4443-bc00-9fc288457d3f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.163316 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4821ddb8e299400eba9adc4bc53ebe6b', 'request_id': 'req-d495ebb7-3fa5-4443-bc00-9fc288457d3f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.169774 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4821ddb8e299400eba9adc4bc53ebe6b', 'request_id': 'req-d495ebb7-3fa5-4443-bc00-9fc288457d3f'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.183219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] The token cache is being invalidate because project 4821ddb8e299400eba9adc4bc53ebe6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:13:06.183506 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.184163 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.185630 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1456/2908] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:13:06 2022] DELETE /identity/v3/projects/4821ddb8e299400eba9adc4bc53ebe6b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:13:06.189852 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:06.191382 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:06.205372 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db1605-c93a-4109-a5f0-257b619854cf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:06.264980 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:06.265619 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:06.265898 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:06.266168 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] PATH_INFO: `/v3/users/5069d76b8fbe49aca57da73f4495a8cc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:06.267391 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5069d76b8fbe49aca57da73f4495a8cc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:06.269369 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:06.317655 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5069d76b8fbe49aca57da73f4495a8cc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.327236 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5069d76b8fbe49aca57da73f4495a8cc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.328013 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5069d76b8fbe49aca57da73f4495a8cc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.341730 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5069d76b8fbe49aca57da73f4495a8cc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.343836 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:06.350800 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5069d76b8fbe49aca57da73f4495a8cc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.356991 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1453/2909] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 9 18:13:06 2022] DELETE /identity/v3/users/5069d76b8fbe49aca57da73f4495a8cc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:13:06.534445 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Authenticating user token {{(pid=80212) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:13:06.535942 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.fernet_utils [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80212) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:13:06.554168 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495ebb7-3fa5-4443-bc00-9fc288457d3f admin admin] Validating token access rules against request {{(pid=80212) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:13:06.580839 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:06.620945 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654dd146616b4cbd8975ee15d286d26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5aa4b373e99e4d74b2148fd834c1abbb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80212) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:13:06.621559 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:13:06.621830 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:13:06.622096 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] PATH_INFO: `/v3/projects/4c240e3a80e2484695d991b45538ff4c` {{(pid=80212) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:13:06.623219 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c240e3a80e2484695d991b45538ff4c)` {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:13:06.625104 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] RBAC: Authorization granted {{(pid=80212) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:13:06.671197 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c240e3a80e2484695d991b45538ff4c', 'request_id': 'req-e67ca794-7e36-4225-bd3d-687624ca9364'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.676617 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c240e3a80e2484695d991b45538ff4c', 'request_id': 'req-e67ca794-7e36-4225-bd3d-687624ca9364'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.683364 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c240e3a80e2484695d991b45538ff4c', 'request_id': 'req-e67ca794-7e36-4225-bd3d-687624ca9364'} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.696587 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] The token cache is being invalidate because project 4c240e3a80e2484695d991b45538ff4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80212) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 18:13:06.696872 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.697515 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG keystone.notifications [None req-e67ca794-7e36-4225-bd3d-687624ca9364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80212) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 18:13:06.699053 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: [pid: 80212|app: 0|req: 1457/2910] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 9 18:13:06 2022] DELETE /identity/v3/projects/4c240e3a80e2484695d991b45538ff4c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 18:13:16.353650 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:13:16.693082 dv-cinder-836083-11-iscsi devstack@keystone.service[80212]: DEBUG dbcounter [-] [80212] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80212) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 18:14:26.920049 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 18:14:26.921498 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:14:27.061085 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aec4f0-b7ca-4486-a62c-ed4d31a1a512 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 18:14:27.121091 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b69ade20aca4860a575feb79522e9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a67b49d3c33443589b9d7970888f1e4d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 18:14:27.121698 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 18:14:27.121976 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 18:14:27.122243 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 18:14:27.124055 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 18:14:27.135287 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 18:14:27.136856 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8c16-f9fd-429c-96ae-1e47c3b0fd74 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 18:14:27.146633 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1454/2911] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 9 18:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 18:14:37.142343 dv-cinder-836083-11-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}